00002000:00010000:2.0:1789760683.535133:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19234816, mtime: 0x6aad94aa, atime: 0x0, ctime: 0x6aad94aa, blocks: 0x9807 00010000:00010000:2.0:1789760683.535142:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544834 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.535158:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=667s v=(0 0) 00010000:00010000:2.0:1789760683.535162:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760683.535168:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544834 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.535182:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544834 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760683.612110:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19234816 -> 19238912 00010000:00010000:2.0:1789760683.612126:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544834 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.612138:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544834 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.612147:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544834 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.612153:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760683.612157:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544834 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.612165:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544834 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.612189:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544834 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.612196:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544834 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760683.612327:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495677184:12345-192.168.203.9@tcp:4:dd.0 Request processed in 77237us (77508us total) trans 4294972013 rc 0/0 00000100:00100000:2.0:1789760683.612336:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4725 00000100:00100000:2.0:1789760683.612346:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 667803163537 time 667855791241 state 0x2 ref 0 00000100:00100000:2.0:1789760683.711044:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495677440 00000100:00100000:2.0:1789760683.711054:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495677440 00000100:00100000:2.0:1789760683.711109:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 667856051657 time 667942971874 state 0x0 ref 1 00000100:00100000:2.0:1789760683.711125:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 667943232290 time 667942971874 state 0x1 ref 1 00000100:00100000:2.0:1789760683.711160:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 667943232290 time 668031877031 state 0x0 ref 1 00000100:00100000:2.0:1789760683.711172:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495677440:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760683.711221:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19243008, mtime: 0x6aad94ab, atime: 0x0, ctime: 0x6aad94ab, blocks: 0x9807 00010000:00010000:2.0:1789760683.711230:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544842 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.711245:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=668s v=(0 0) 00010000:00010000:2.0:1789760683.711250:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760683.711256:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544842 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19243008->19247103] (req 19243008->19247103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.711269:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544842 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19243008->19247103] (req 19243008->19247103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760683.770218:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495677568 00000100:00100000:2.0:1789760683.770228:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495677568 00000100:00100000:2.0:1789760683.770547:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 667943232290 time 668031877031 state 0x0 ref 1 00000100:00100000:2.0:1789760683.770567:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 668032137447 time 668031877031 state 0x1 ref 1 00000100:00100000:2.0:1789760683.770637:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 668032137447 time 668091352031 state 0x0 ref 1 00000100:00100000:2.0:1789760683.770651:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495677568:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760683.770704:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19247104, mtime: 0x6aad94ab, atime: 0x0, ctime: 0x6aad94ab, blocks: 0x9807 00010000:00010000:2.0:1789760683.770715:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544849 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.770733:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=668s v=(0 0) 00010000:00010000:2.0:1789760683.770748:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760683.770755:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544849 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19247104->19251199] (req 19247104->19251199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.770769:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544849 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19247104->19251199] (req 19247104->19251199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760683.858662:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495677824 00000100:00100000:2.0:1789760683.858673:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495677824 00000100:00100000:2.0:1789760683.859770:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 668091612447 time 668128378957 state 0x0 ref 1 00000100:00100000:2.0:1789760683.859786:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 668128639373 time 668128378957 state 0x1 ref 1 00000100:00100000:2.0:1789760683.859891:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 668128639373 time 668180606350 state 0x0 ref 1 00000100:00100000:2.0:1789760683.859905:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495677824:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760683.859958:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19255296, mtime: 0x6aad94ab, atime: 0x0, ctime: 0x6aad94ab, blocks: 0x9807 00010000:00010000:2.0:1789760683.859967:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544857 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.859983:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=668s v=(0 0) 00010000:00010000:2.0:1789760683.859988:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760683.859994:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544857 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19255296->19259391] (req 19255296->19259391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760683.860007:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544857 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19255296->19259391] (req 19255296->19259391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760683.923200:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495677952 00000100:00100000:2.0:1789760683.923210:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495677952 00000100:00100000:2.0:1789760683.923266:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 668128639373 time 668180606350 state 0x0 ref 1 00000100:00100000:2.0:1789760683.923377:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 668180866766 time 668180606350 state 0x1 ref 1 00000001:00080000:2.0:1789760684.122706:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972020 is committed 00000100:00100000:2.0:1789760684.186520:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495678336 00000100:00100000:2.0:1789760684.186530:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495678336 00000100:00100000:2.0:1789760684.186582:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 668340287988 time 668447890394 state 0x0 ref 1 00000100:00100000:2.0:1789760684.186597:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 668448150810 time 668447890394 state 0x1 ref 1 00000100:00100000:2.0:1789760684.186628:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 668448150810 time 668507345175 state 0x0 ref 1 00000100:00100000:2.0:1789760684.186641:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495678336:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760684.186687:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19271680, mtime: 0x6aad94ab, atime: 0x0, ctime: 0x6aad94ab, blocks: 0x9807 00010000:00010000:2.0:1789760684.186695:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544873 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.186712:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=668s v=(0 0) 00010000:00010000:2.0:1789760684.186716:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760684.186721:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544873 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.186734:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544873 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760684.235175:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19271680 -> 19275776 00010000:00010000:2.0:1789760684.235190:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544873 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.235209:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544873 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.235217:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544873 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.235224:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760684.235228:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544873 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.235237:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544873 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.235255:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544873 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.235262:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544873 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760684.235393:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495678336:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48749us (49129us total) trans 4294972022 rc 0/0 00000100:00100000:2.0:1789760684.235403:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4734 00000100:00100000:2.0:1789760684.235412:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 668448150810 time 668507345175 state 0x2 ref 0 00000100:00100000:2.0:1789760684.240030:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:2.0:1789760684.289761:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972023 is committed 00000100:00100000:2.0:1789760684.293945:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495678592 00000100:00100000:2.0:1789760684.293955:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495678592 00000100:00100000:2.0:1789760684.294008:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 668507605591 time 668560940277 state 0x0 ref 1 00000100:00100000:2.0:1789760684.294023:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 668561200693 time 668560940277 state 0x1 ref 1 00000100:00100000:2.0:1789760684.294058:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 668561200693 time 668614774229 state 0x0 ref 1 00000100:00100000:2.0:1789760684.294069:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495678592:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760684.294117:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19279872, mtime: 0x6aad94ab, atime: 0x0, ctime: 0x6aad94ab, blocks: 0x9807 00010000:00010000:2.0:1789760684.294138:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544881 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.294153:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=668s v=(0 0) 00010000:00010000:2.0:1789760684.294158:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760684.294164:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544881 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.294176:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544881 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760684.345681:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972024 is committed 00000100:00100000:2.0:1789760684.356907:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495678720 00000100:00100000:2.0:1789760684.356917:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495678720 00000100:00100000:2.0:1789760684.356971:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 668561200693 time 668614774229 state 0x0 ref 1 00000100:00100000:2.0:1789760684.356985:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 668615034645 time 668614774229 state 0x1 ref 1 00000100:00100000:2.0:1789760684.357019:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 668615034645 time 668677736283 state 0x0 ref 1 00000100:00100000:2.0:1789760684.357031:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495678720:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760684.357078:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19283968, mtime: 0x6aad94ab, atime: 0x0, ctime: 0x6aad94ab, blocks: 0x9807 00010000:00010000:2.0:1789760684.357087:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354544888 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.357104:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=668s v=(0 0) 00010000:00010000:2.0:1789760684.357108:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760684.357114:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354544888 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19283968->19288063] (req 19283968->19288063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.357128:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354544888 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19283968->19288063] (req 19283968->19288063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760684.408921:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972025 is committed 00000100:00100000:2.0:1789760684.418821:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760684.470965:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00002000:00010000:2.0:1789760684.508111:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19292160 -> 19296256 00010000:00010000:2.0:1789760684.508126:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544896 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.508139:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544896 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.508147:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544896 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.508156:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760684.508161:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544896 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.508169:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544896 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.508188:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544896 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.508195:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544896 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760684.508543:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92902dbc4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495678976:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37032us (37324us total) trans 4294972027 rc 0/0 00000100:00100000:2.0:1789760684.508561:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4739 00000100:00100000:2.0:1789760684.508687:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 668739969650 time 668791952778 state 0x2 ref 0 00000100:00100000:2.0:1789760684.601647:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495679360 00000100:00100000:2.0:1789760684.601663:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495679360 00000100:00100000:2.0:1789760684.601723:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 668835348106 time 668889390035 state 0x0 ref 1 00000100:00100000:2.0:1789760684.601740:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 668889650451 time 668889390035 state 0x1 ref 1 00000100:00100000:2.0:1789760684.601770:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 668889650451 time 668922486823 state 0x0 ref 1 00000100:00100000:2.0:1789760684.601786:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495679360:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760684.601832:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19304448, mtime: 0x6aad94ab, atime: 0x0, ctime: 0x6aad94ab, blocks: 0x9807 00010000:00010000:2.0:1789760684.601840:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545448ab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.601856:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=668s v=(0 0) 00010000:00010000:2.0:1789760684.601860:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760684.601865:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545448ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760684.601878:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545448ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760684.688172:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760684.758136:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760684.843055:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00010000:00010000:2.0:1789760684.897032:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789760684.930519:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:2.0:1789760685.007613:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972034 is committed 00000100:00100000:2.0:1789760685.011935:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760685.063983:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760685.130035:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00002000:00010000:2.0:1789760685.177030:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19333120 -> 19337216 00010000:00010000:2.0:1789760685.177050:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545448dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760685.177069:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545448dc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760685.177078:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545448dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760685.177085:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760685.177090:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545448dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760685.177100:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545448dc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760685.177119:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545448dc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760685.177127:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545448dc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760685.177220:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495680256:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46960us (47190us total) trans 4294972037 rc 0/0 00000100:00100000:2.0:1789760685.177230:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4749 00000100:00100000:2.0:1789760685.177240:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 669385151310 time 669450961452 state 0x2 ref 0 00000100:00100000:2.0:1789760685.187518:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760685.298366:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00002000:00010000:2.0:1789760685.369624:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19345408 -> 19349504 00010000:00010000:2.0:1789760685.369643:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545448f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19345408->19349503] (req 19345408->19349503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760685.369657:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545448f1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19345408->19349503] (req 19345408->19349503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760685.369666:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545448f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19345408->19349503] (req 19345408->19349503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760685.369673:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760685.369678:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545448f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19345408->19349503] (req 19345408->19349503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760685.369688:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545448f1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19345408->19349503] (req 19345408->19349503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760685.369707:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545448f1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19345408->19349503] (req 19345408->19349503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760685.369714:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545448f1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19345408->19349503] (req 19345408->19349503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760685.369851:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495680640:12345-192.168.203.9@tcp:4:dd.0 Request processed in 71140us (71481us total) trans 4294972040 rc 0/0 00000100:00100000:2.0:1789760685.369862:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4752 00000100:00100000:2.0:1789760685.369872:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 669549133092 time 669619404438 state 0x2 ref 0 00000100:00100000:2.0:1789760685.380378:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760685.499992:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495681024 00000100:00100000:2.0:1789760685.500002:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495681024 00000100:00100000:2.0:1789760685.500057:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 669701645870 time 669758801137 state 0x0 ref 1 00000100:00100000:2.0:1789760685.500071:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 669759061553 time 669758801137 state 0x1 ref 1 00000100:00100000:2.0:1789760685.500105:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 669759061553 time 669820822039 state 0x0 ref 1 00000100:00100000:2.0:1789760685.500117:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02768e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495681024:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760685.500163:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19357696, mtime: 0x6aad94ac, atime: 0x0, ctime: 0x6aad94ac, blocks: 0x9807 00010000:00010000:2.0:1789760685.500172:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544906 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760685.500187:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=669s v=(0 0) 00010000:00010000:2.0:1789760685.500191:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760685.500197:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544906 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19357696->19361791] (req 19357696->19361791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760685.500211:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544906 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19357696->19361791] (req 19357696->19361791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760685.672414:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760685.896223:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00010000:00010000:2.0:1789760685.918730:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760685.978110:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972047 is committed 00000100:00100000:2.0:1789760685.986534:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:2.0:1789760686.136826:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972048 is committed 00000100:00100000:2.0:1789760686.144510:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760686.219609:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:2.0:1789760686.493194:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972052 is committed 00000100:00100000:2.0:1789760686.502747:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:2.0:1789760686.561191:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972053 is committed 00000001:00080000:2.0:1789760686.594941:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972054 is committed 00000100:00100000:2.0:1789760686.638096:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760686.834215:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760686.882170:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f01ac5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520296576:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760686.882237:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00010000:2.0:1789760686.965259:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19431424 -> 19435520 00010000:00010000:2.0:1789760686.965276:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354544984 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.965315:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354544984 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.965324:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354544984 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.965331:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760686.965336:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354544984 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.965345:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354544984 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.965365:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354544984 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760686.965372:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354544984 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760686.965467:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495683328:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46074us (51980us total) trans 4294972061 rc 0/0 00000100:00100000:2.0:1789760686.965477:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4773 00000100:00100000:2.0:1789760686.965486:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 671155476346 time 671240095862 state 0x2 ref 0 00002000:00010000:2.0:1789760687.151734:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19439616 -> 19443712 00010000:00010000:2.0:1789760687.151752:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544992 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.151765:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544992 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.151842:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544992 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.151850:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760687.151855:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544992 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.151869:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544992 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.151888:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544992 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.151897:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544992 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760687.152040:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495683584:12345-192.168.203.9@tcp:4:dd.0 Request processed in 81681us (81997us total) trans 4294972063 rc 0/0 00000100:00100000:2.0:1789760687.152051:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4775 00000100:00100000:2.0:1789760687.152061:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 671294782451 time 671391060081 state 0x2 ref 0 00000100:00100000:2.0:1789760687.173266:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495683712 00000100:00100000:2.0:1789760687.173275:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495683712 00000100:00100000:2.0:1789760687.173348:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 671294782451 time 671391060081 state 0x0 ref 1 00000100:00100000:2.0:1789760687.173362:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 671391320497 time 671391060081 state 0x1 ref 1 00000100:00100000:2.0:1789760687.173394:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 671391320497 time 671494110613 state 0x0 ref 1 00000100:00100000:2.0:1789760687.173407:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495683712:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760687.173474:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19443712, mtime: 0x6aad94ae, atime: 0x0, ctime: 0x6aad94ae, blocks: 0x9807 00010000:00010000:2.0:1789760687.173483:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544999 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.173500:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=671s v=(0 0) 00010000:00010000:2.0:1789760687.173504:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760687.173511:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544999 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.173524:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544999 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760687.254333:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19443712 -> 19447808 00010000:00010000:2.0:1789760687.254348:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544999 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.254360:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544999 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.254368:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544999 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.254375:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760687.254379:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544999 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.254388:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544999 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.254407:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544999 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.254415:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544999 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760687.254517:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495683712:12345-192.168.203.9@tcp:4:dd.0 Request processed in 81108us (81353us total) trans 4294972064 rc 0/0 00000100:00100000:2.0:1789760687.254528:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4776 00000100:00100000:2.0:1789760687.254538:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 671391320497 time 671494110613 state 0x2 ref 0 00000001:00080000:2.0:1789760687.320466:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972065 is committed 00000100:00100000:2.0:1789760687.330162:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495683968 00000100:00100000:2.0:1789760687.330175:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495683968 00000100:00100000:2.0:1789760687.330236:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 671494371029 time 671578855391 state 0x0 ref 1 00000100:00100000:2.0:1789760687.330251:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 671579115807 time 671578855391 state 0x1 ref 1 00000100:00100000:2.0:1789760687.330292:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 671579115807 time 671650993623 state 0x0 ref 1 00000100:00100000:2.0:1789760687.330308:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495683968:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760687.330359:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19451904, mtime: 0x6aad94ae, atime: 0x0, ctime: 0x6aad94ae, blocks: 0x9807 00010000:00010000:2.0:1789760687.330368:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545449a7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.330385:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=671s v=(0 0) 00010000:00010000:2.0:1789760687.330390:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760687.330397:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545449a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.330412:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545449a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760687.413535:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19451904 -> 19456000 00010000:00010000:2.0:1789760687.413550:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545449a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.413563:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545449a7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.413573:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545449a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.413580:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760687.413584:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545449a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.413593:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545449a7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.413614:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545449a7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.413622:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545449a7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760687.413724:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495683968:12345-192.168.203.9@tcp:4:dd.0 Request processed in 83415us (85942us total) trans 4294972066 rc 0/0 00000100:00100000:2.0:1789760687.413735:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4778 00000100:00100000:2.0:1789760687.413746:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 671579115807 time 671650993623 state 0x2 ref 0 00000100:00100000:2.0:1789760687.424413:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00002000:00010000:2.0:1789760687.467239:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19456000 -> 19460096 00010000:00010000:2.0:1789760687.467260:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545449ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19456000->19460095] (req 19456000->19460095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.467274:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545449ae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19456000->19460095] (req 19456000->19460095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.467314:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545449ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19456000->19460095] (req 19456000->19460095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.467322:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760687.467328:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545449ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19456000->19460095] (req 19456000->19460095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.467338:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545449ae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19456000->19460095] (req 19456000->19460095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.467358:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545449ae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19456000->19460095] (req 19456000->19460095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.467373:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545449ae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19456000->19460095] (req 19456000->19460095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760687.467491:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495684096:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42839us (43081us total) trans 4294972067 rc 0/0 00000100:00100000:2.0:1789760687.467502:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4779 00000100:00100000:2.0:1789760687.467512:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 671651254039 time 671745352266 state 0x2 ref 0 00000001:00080000:2.0:1789760687.501113:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972068 is committed 00000100:00100000:2.0:1789760687.505350:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495684352 00000100:00100000:2.0:1789760687.505359:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495684352 00000100:00100000:2.0:1789760687.505411:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 671745612682 time 671791579765 state 0x0 ref 1 00000100:00100000:2.0:1789760687.505493:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 671791840181 time 671791579765 state 0x1 ref 1 00000100:00100000:2.0:1789760687.505549:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 671791840181 time 671826265386 state 0x0 ref 1 00000100:00100000:2.0:1789760687.505562:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd01c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495684352:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760687.505610:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19464192, mtime: 0x6aad94ae, atime: 0x0, ctime: 0x6aad94ae, blocks: 0x9807 00010000:00010000:2.0:1789760687.505620:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545449bc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.505637:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=671s v=(0 0) 00010000:00010000:2.0:1789760687.505641:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760687.505647:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545449bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.505662:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545449bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760687.538185:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495684480 00000100:00100000:2.0:1789760687.538194:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495684480 00000100:00100000:2.0:1789760687.538249:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 671791840181 time 671826265386 state 0x0 ref 1 00000100:00100000:2.0:1789760687.538263:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 671826525802 time 671826265386 state 0x1 ref 1 00000100:00100000:2.0:1789760687.538329:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 671826525802 time 671859045482 state 0x0 ref 1 00000100:00100000:2.0:1789760687.538342:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495684480:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760687.538390:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19468288, mtime: 0x6aad94ae, atime: 0x0, ctime: 0x6aad94ae, blocks: 0x9807 00010000:00010000:2.0:1789760687.538399:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545449c3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.538414:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=671s v=(0 0) 00010000:00010000:2.0:1789760687.538419:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760687.538437:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545449c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19468288->19472383] (req 19468288->19472383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.538459:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545449c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19468288->19472383] (req 19468288->19472383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760687.567919:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:2.0:1789760687.596666:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972071 is committed 00000100:00100000:2.0:1789760687.600321:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760687.604088:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495684864 00000100:00100000:2.0:1789760687.604093:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495684864 00000100:00100000:2.0:1789760687.604135:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 380 00000100:00100000:2.0:1789760687.604142:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290062f4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2356:x1876699495684864:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:2.0:1789760687.604178:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290062f4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2356:x1876699495684864:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 37us (795us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760687.604184:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 380 00000100:00100000:2.0:1789760687.636128:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495685248 00000100:00100000:2.0:1789760687.636138:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495685248 00000100:00100000:2.0:1789760687.636191:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 671889123998 time 671921384686 state 0x0 ref 1 00000100:00100000:2.0:1789760687.636207:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 671921645102 time 671921384686 state 0x1 ref 1 00000100:00100000:2.0:1789760687.636318:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 671921645102 time 671957033421 state 0x0 ref 1 00000100:00100000:2.0:1789760687.636331:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495685248:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760687.636377:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19480576, mtime: 0x6aad94ae, atime: 0x0, ctime: 0x6aad94ae, blocks: 0x9807 00010000:00010000:2.0:1789760687.636387:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545449d8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.636403:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=671s v=(0 0) 00010000:00010000:2.0:1789760687.636408:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760687.636414:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545449d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.636580:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545449d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760687.662063:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972073 is committed 00000001:00080000:2.0:1789760687.698857:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972074 is committed 00000100:00100000:2.0:1789760687.707043:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:2.0:1789760687.829589:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972077 is committed 00000100:00100000:2.0:1789760687.834961:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495685888 00000100:00100000:2.0:1789760687.834971:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495685888 00000100:00100000:2.0:1789760687.835022:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 672068969823 time 672111949635 state 0x0 ref 1 00000100:00100000:2.0:1789760687.835037:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 672112210051 time 672111949635 state 0x1 ref 1 00000100:00100000:2.0:1789760687.835064:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 672112210051 time 672155781642 state 0x0 ref 1 00000100:00100000:2.0:1789760687.835077:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495685888:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760687.835120:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19501056, mtime: 0x6aad94af, atime: 0x0, ctime: 0x6aad94af, blocks: 0x9807 00010000:00010000:2.0:1789760687.835128:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545449fb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.835143:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=672s v=(0 0) 00010000:00010000:2.0:1789760687.835147:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760687.835153:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545449fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760687.835166:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545449fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760687.863629:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760687.938368:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:2.0:1789760687.974692:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972080 is committed 00002000:00010000:2.0:1789760688.010875:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19513344 -> 19517440 00010000:00010000:2.0:1789760688.010889:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544a10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19513344->19517439] (req 19513344->19517439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.010902:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544a10 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19513344->19517439] (req 19513344->19517439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.010911:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544a10 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19513344->19517439] (req 19513344->19517439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.010918:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760688.010932:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544a10 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19513344->19517439] (req 19513344->19517439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.010942:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544a10 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19513344->19517439] (req 19513344->19517439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.010961:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544a10 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19513344->19517439] (req 19513344->19517439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.010969:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544a10 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19513344->19517439] (req 19513344->19517439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760688.011251:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495686272:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31488us (31799us total) trans 4294972081 rc 0/0 00000100:00100000:2.0:1789760688.011263:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4793 00000100:00100000:2.0:1789760688.011273:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 672259669165 time 672300459047 state 0x2 ref 0 00000001:00080000:2.0:1789760688.082504:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972082 is committed 00002000:00010000:2.0:1789760688.156987:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19521536 -> 19525632 00010000:00010000:2.0:1789760688.157003:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354544a1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.157017:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354544a1e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.157026:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354544a1e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.157033:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760688.157038:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354544a1e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.157049:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354544a1e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.157069:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354544a1e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.157083:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354544a1e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760688.157182:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495686528:12345-192.168.203.9@tcp:4:dd.0 Request processed in 60153us (60403us total) trans 4294972083 rc 0/0 00000100:00100000:2.0:1789760688.157193:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4795 00000100:00100000:2.0:1789760688.157203:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 672340321877 time 672417727296 state 0x2 ref 0 00000100:00100000:2.0:1789760688.162184:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495686656 00000100:00100000:2.0:1789760688.162194:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495686656 00000100:00100000:2.0:1789760688.162247:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 672340321877 time 672417727296 state 0x0 ref 1 00000100:00100000:2.0:1789760688.162262:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 672417987712 time 672417727296 state 0x1 ref 1 00000100:00100000:2.0:1789760688.162304:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 672417987712 time 672483021345 state 0x0 ref 1 00000100:00100000:2.0:1789760688.162316:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495686656:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760688.162363:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19525632, mtime: 0x6aad94af, atime: 0x0, ctime: 0x6aad94af, blocks: 0x9807 00010000:00010000:2.0:1789760688.162373:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544a25 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.162389:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=672s v=(0 0) 00010000:00010000:2.0:1789760688.162394:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760688.162400:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544a25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19525632->19529727] (req 19525632->19529727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.162429:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544a25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19525632->19529727] (req 19525632->19529727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760688.194150:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19525632 -> 19529728 00010000:00010000:2.0:1789760688.194167:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544a25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19525632->19529727] (req 19525632->19529727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.194179:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544a25 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19525632->19529727] (req 19525632->19529727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.194188:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544a25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19525632->19529727] (req 19525632->19529727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.194195:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760688.194200:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544a25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19525632->19529727] (req 19525632->19529727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.194208:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544a25 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19525632->19529727] (req 19525632->19529727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.194227:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544a25 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19525632->19529727] (req 19525632->19529727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.194234:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544a25 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19525632->19529727] (req 19525632->19529727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760688.194362:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495686656:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32042us (32424us total) trans 4294972084 rc 0/0 00000100:00100000:2.0:1789760688.194373:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4796 00000100:00100000:2.0:1789760688.194384:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 672417987712 time 672483021345 state 0x2 ref 0 00000001:00080000:2.0:1789760688.243757:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972085 is committed 00000100:00100000:2.0:1789760688.250068:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495686912 00000100:00100000:2.0:1789760688.250078:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495686912 00000100:00100000:2.0:1789760688.250206:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 672483281761 time 672519690623 state 0x0 ref 1 00000100:00100000:2.0:1789760688.250221:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 672519951039 time 672519690623 state 0x1 ref 1 00000100:00100000:2.0:1789760688.250310:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 672519951039 time 672571026437 state 0x0 ref 1 00000100:00100000:2.0:1789760688.250323:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495686912:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760688.250367:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19533824, mtime: 0x6aad94af, atime: 0x0, ctime: 0x6aad94af, blocks: 0x9807 00010000:00010000:2.0:1789760688.250375:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544a33 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.250390:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=672s v=(0 0) 00010000:00010000:2.0:1789760688.250395:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760688.250402:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544a33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.250414:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544a33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760688.303860:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495687040 00000100:00100000:2.0:1789760688.303871:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495687040 00000100:00100000:2.0:1789760688.303947:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 672519951039 time 672571026437 state 0x0 ref 1 00000100:00100000:2.0:1789760688.303965:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 672571286853 time 672571026437 state 0x1 ref 1 00000100:00100000:2.0:1789760688.304008:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 672571286853 time 672624724074 state 0x0 ref 1 00000100:00100000:2.0:1789760688.304024:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495687040:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760688.304078:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19537920, mtime: 0x6aad94af, atime: 0x0, ctime: 0x6aad94af, blocks: 0x9807 00010000:00010000:2.0:1789760688.304088:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354544a3a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.304116:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=672s v=(0 0) 00010000:00010000:2.0:1789760688.304121:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760688.304128:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354544a3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19537920->19542015] (req 19537920->19542015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.304142:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354544a3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19537920->19542015] (req 19537920->19542015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760688.456867:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495687296 00000100:00100000:2.0:1789760688.456878:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495687296 00000100:00100000:2.0:1789760688.456932:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 672624984490 time 672697584832 state 0x0 ref 1 00000100:00100000:2.0:1789760688.456947:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 672697845248 time 672697584832 state 0x1 ref 1 00000100:00100000:2.0:1789760688.456978:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 672697845248 time 672777694901 state 0x0 ref 1 00000100:00100000:2.0:1789760688.456990:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04577b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495687296:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760688.457039:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19546112, mtime: 0x6aad94af, atime: 0x0, ctime: 0x6aad94af, blocks: 0x9807 00010000:00010000:2.0:1789760688.457047:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354544a48 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.457062:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=672s v=(0 0) 00010000:00010000:2.0:1789760688.457067:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760688.457073:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354544a48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.457087:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354544a48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760688.725969:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972092 is committed 00000001:00080000:2.0:1789760688.753874:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972093 is committed 00000001:00080000:2.0:1789760688.805667:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972095 is committed 00000100:00100000:2.0:1789760688.839072:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:2.0:1789760688.864220:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972097 is committed 00000001:00080000:2.0:1789760688.931875:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972098 is committed 00002000:00010000:2.0:1789760688.975558:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19587072 -> 19591168 00010000:00010000:2.0:1789760688.975575:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354544a8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19587072->19591167] (req 19587072->19591167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.975587:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354544a8e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19587072->19591167] (req 19587072->19591167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.975597:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354544a8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19587072->19591167] (req 19587072->19591167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.975603:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760688.975608:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354544a8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19587072->19591167] (req 19587072->19591167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.975616:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354544a8e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19587072->19591167] (req 19587072->19591167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.975638:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354544a8e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19587072->19591167] (req 19587072->19591167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760688.975645:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354544a8e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19587072->19591167] (req 19587072->19591167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760688.975756:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495688576:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36945us (37357us total) trans 4294972099 rc 0/0 00000100:00100000:2.0:1789760688.975767:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4811 00000100:00100000:2.0:1789760688.975783:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 673190243673 time 673259509274 state 0x2 ref 0 00000100:00100000:2.0:1789760688.988236:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:2.0:1789760689.031254:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972100 is committed 00000100:00100000:2.0:1789760689.036199:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760689.036900:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 673259769690 time 673309304796 state 0x0 ref 1 00000100:00100000:2.0:1789760689.036916:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 673309565212 time 673309304796 state 0x1 ref 1 00000100:00100000:2.0:1789760689.036944:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 673309565212 time 673357661355 state 0x0 ref 1 00000100:00100000:2.0:1789760689.036956:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290363a1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495688832:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760689.037010:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19595264, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0x9807 00010000:00010000:2.0:1789760689.037020:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544a9c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.037036:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=673s v=(0 0) 00010000:00010000:2.0:1789760689.037040:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760689.037047:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544a9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.037061:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544a9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760689.115160:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972101 is committed 00000100:00100000:2.0:1789760689.127022:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495688960 00000100:00100000:2.0:1789760689.127036:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495688960 00000100:00100000:2.0:1789760689.127094:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 673309565212 time 673357661355 state 0x0 ref 1 00000100:00100000:2.0:1789760689.127109:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 673357921771 time 673357661355 state 0x1 ref 1 00000100:00100000:2.0:1789760689.127138:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 673357921771 time 673447855644 state 0x0 ref 1 00000100:00100000:2.0:1789760689.127153:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495688960:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760689.127200:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19599360, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0x9807 00010000:00010000:2.0:1789760689.127208:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544aa3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.127223:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=673s v=(0 0) 00010000:00010000:2.0:1789760689.127227:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760689.127233:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544aa3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.127248:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544aa3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760689.205368:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19599360 -> 19603456 00010000:00010000:2.0:1789760689.205382:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544aa3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.205394:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544aa3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.205404:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544aa3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.205411:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760689.205416:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544aa3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.205446:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544aa3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.205466:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544aa3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.205474:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544aa3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760689.205559:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495688960:12345-192.168.203.9@tcp:4:dd.0 Request processed in 78404us (84427us total) trans 4294972102 rc 0/0 00000100:00100000:2.0:1789760689.205569:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4814 00000100:00100000:2.0:1789760689.205578:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 673357921771 time 673447855644 state 0x2 ref 0 00000100:00100000:2.0:1789760689.210365:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760689.270214:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495689216 00000100:00100000:2.0:1789760689.270223:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495689216 00000100:00100000:2.0:1789760689.270276:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 673448116060 time 673531746766 state 0x0 ref 1 00000100:00100000:2.0:1789760689.270319:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 673532007182 time 673531746766 state 0x1 ref 1 00000100:00100000:2.0:1789760689.270347:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 673532007182 time 673591063739 state 0x0 ref 1 00000100:00100000:2.0:1789760689.270359:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495689216:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760689.270405:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19607552, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0x9807 00010000:00010000:2.0:1789760689.270627:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544ab1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.270645:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=673s v=(0 0) 00010000:00010000:2.0:1789760689.270650:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760689.270657:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544ab1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.270670:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544ab1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760689.347977:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760689.439729:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520296704:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760689.439805:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1789760689.442754:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760689.442820:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520296704:0@lo:13:osp-pre-0-0.0 00002000:00010000:2.0:1789760689.508255:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19619840 -> 19623936 00010000:00010000:2.0:1789760689.508270:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544ac6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.512087:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544ac6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.512100:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544ac6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.512109:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760689.512114:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544ac6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.512125:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544ac6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.512148:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544ac6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.512157:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544ac6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760689.512323:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495689600:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39932us (40288us total) trans 4294972107 rc 0/0 00000100:00100000:2.0:1789760689.512334:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4819 00000100:00100000:2.0:1789760689.512344:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 673727145692 time 673793048776 state 0x2 ref 0 00000100:00100000:2.0:1789760689.518951:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00002000:00010000:2.0:1789760689.569201:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19623936 -> 19628032 00010000:00010000:2.0:1789760689.569217:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544acd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.569237:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544acd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.569246:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544acd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.569258:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760689.569263:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544acd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.569272:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544acd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.569315:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544acd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.569325:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544acd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760689.569402:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92901237ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495689728:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50221us (50459us total) trans 4294972108 rc 0/0 00000100:00100000:2.0:1789760689.571453:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4820 00000100:00100000:2.0:1789760689.571465:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 673793309192 time 673839883831 state 0x2 ref 0 00000100:00100000:2.0:1789760689.640017:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495689984 00000100:00100000:2.0:1789760689.640028:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495689984 00000100:00100000:2.0:1789760689.640083:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 673840144247 time 673897692799 state 0x0 ref 1 00000100:00100000:2.0:1789760689.640098:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 673897953215 time 673897692799 state 0x1 ref 1 00000100:00100000:2.0:1789760689.640130:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 673897953215 time 673960846940 state 0x0 ref 1 00000100:00100000:2.0:1789760689.640145:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495689984:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760689.640192:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19632128, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0x9807 00010000:00010000:2.0:1789760689.640200:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544adb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.640215:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=673s v=(0 0) 00010000:00010000:2.0:1789760689.640220:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760689.640226:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544adb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.640240:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544adb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760689.730147:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972110 is committed 00000100:00100000:2.0:1789760689.739880:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495690112 00000100:00100000:2.0:1789760689.739893:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495690112 00000100:00100000:2.0:1789760689.739969:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 673897953215 time 673960846940 state 0x0 ref 1 00000100:00100000:2.0:1789760689.739984:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 673961107356 time 673960846940 state 0x1 ref 1 00000100:00100000:2.0:1789760689.740017:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 673961107356 time 674060733950 state 0x0 ref 1 00000100:00100000:2.0:1789760689.740030:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01c8c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495690112:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760689.740078:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19636224, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0x9807 00010000:00010000:2.0:1789760689.740086:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544ae2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.740102:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=674s v=(0 0) 00010000:00010000:2.0:1789760689.740107:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760689.740112:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544ae2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.740126:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544ae2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760689.801944:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00002000:00010000:2.0:1789760689.849699:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19640320 -> 19644416 00010000:00010000:2.0:1789760689.849716:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354544ae9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.849728:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354544ae9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.849738:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354544ae9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.849755:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760689.849760:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354544ae9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.849770:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354544ae9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.849794:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354544ae9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.849801:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354544ae9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760689.849936:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fdc9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495690240:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47693us (47995us total) trans 4294972112 rc 0/0 00000100:00100000:2.0:1789760689.849947:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4824 00000100:00100000:2.0:1789760689.849957:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 674060994366 time 674122944173 state 0x2 ref 0 00000100:00100000:2.0:1789760689.855850:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:2.0:1789760689.914336:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972113 is committed 00000100:00100000:2.0:1789760689.918668:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495690496 00000100:00100000:2.0:1789760689.918677:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495690496 00000100:00100000:2.0:1789760689.918731:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 674123204589 time 674176917176 state 0x0 ref 1 00000100:00100000:2.0:1789760689.918745:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 674177177592 time 674176917176 state 0x1 ref 1 00000100:00100000:2.0:1789760689.918783:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 674177177592 time 674239499244 state 0x0 ref 1 00000100:00100000:2.0:1789760689.918795:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01c8d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495690496:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760689.918844:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19648512, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0x9807 00010000:00010000:2.0:1789760689.918853:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544af7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.918873:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=674s v=(0 0) 00010000:00010000:2.0:1789760689.918878:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760689.918884:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544af7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.918897:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544af7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760689.976616:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495690624 00000100:00100000:2.0:1789760689.976625:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495690624 00000100:00100000:2.0:1789760689.976678:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 674177177592 time 674239499244 state 0x0 ref 1 00000100:00100000:2.0:1789760689.976693:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 674239759660 time 674239499244 state 0x1 ref 1 00000100:00100000:2.0:1789760689.976763:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 674239759660 time 674297478515 state 0x0 ref 1 00000100:00100000:2.0:1789760689.976776:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f873480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495690624:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760689.976822:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19652608, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0x9807 00010000:00010000:2.0:1789760689.976832:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544afe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.976848:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=674s v=(0 0) 00010000:00010000:2.0:1789760689.976853:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760689.976859:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544afe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760689.976879:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544afe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760690.016358:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19652608 -> 19656704 00010000:00010000:2.0:1789760690.016379:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544afe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.016393:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544afe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.016403:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544afe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.017437:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760690.017444:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544afe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.017455:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544afe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.017477:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544afe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.017484:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544afe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760690.017620:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f873480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495690624:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40842us (41118us total) trans 4294972115 rc 0/0 00000100:00100000:2.0:1789760690.017631:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4827 00000100:00100000:2.0:1789760690.017641:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 674239759660 time 674297478515 state 0x2 ref 0 00000100:00100000:2.0:1789760690.073002:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495690880 00000100:00100000:2.0:1789760690.073016:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495690880 00000100:00100000:2.0:1789760690.125074:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495691008 00000100:00100000:2.0:1789760690.125084:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495691008 00000100:00100000:2.0:1789760690.125138:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 674349432295 time 674393971715 state 0x0 ref 1 00000100:00100000:2.0:1789760690.125153:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 674394232131 time 674393971715 state 0x1 ref 1 00000100:00100000:2.0:1789760690.125184:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 674394232131 time 674445900838 state 0x0 ref 1 00000100:00100000:2.0:1789760690.125197:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495691008:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760690.125245:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19664896, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0x9807 00010000:00010000:2.0:1789760690.125254:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544b13 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.125269:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=674s v=(0 0) 00010000:00010000:2.0:1789760690.125274:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760690.125320:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544b13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.125335:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544b13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760690.185603:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19664896 -> 19668992 00010000:00010000:2.0:1789760690.185620:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544b13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.185633:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544b13 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.185642:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544b13 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.185650:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760690.185654:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544b13 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.185663:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544b13 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.185683:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544b13 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.185691:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544b13 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760690.185792:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495691008:12345-192.168.203.9@tcp:4:dd.0 Request processed in 60593us (60930us total) trans 4294972118 rc 0/0 00000100:00100000:2.0:1789760690.185876:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4830 00000100:00100000:2.0:1789760690.185886:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 674394232131 time 674445900838 state 0x2 ref 0 00000100:00100000:2.0:1789760690.190829:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00002000:00010000:2.0:1789760690.241720:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19668992 -> 19673088 00010000:00010000:2.0:1789760690.241740:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354544b1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.241760:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354544b1a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.241769:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354544b1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.241777:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760690.241783:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354544b1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.241793:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354544b1a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.241813:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354544b1a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.241821:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354544b1a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760690.241920:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f921c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495691136:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50841us (51096us total) trans 4294972119 rc 0/0 00000100:00100000:2.0:1789760690.241931:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4831 00000100:00100000:2.0:1789760690.241940:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 674446161254 time 674511777205 state 0x2 ref 0 00000100:00100000:2.0:1789760690.332519:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495691520 00000100:00100000:2.0:1789760690.332529:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495691520 00000100:00100000:2.0:1789760690.332590:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 674566971288 time 674610325496 state 0x0 ref 1 00000100:00100000:2.0:1789760690.332606:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 674610585912 time 674610325496 state 0x1 ref 1 00000100:00100000:2.0:1789760690.332635:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 674610585912 time 674653351467 state 0x0 ref 1 00000100:00100000:2.0:1789760690.332649:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495691520:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760690.332698:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19681280, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0x9807 00010000:00010000:2.0:1789760690.332708:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544b2f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.332725:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=674s v=(0 0) 00010000:00010000:2.0:1789760690.332730:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760690.332736:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544b2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.332758:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544b2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760690.474928:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:2.0:1789760690.534764:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972125 is committed 00000100:00100000:2.0:1789760690.547691:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:2.0:1789760690.607058:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972126 is committed 00000100:00100000:2.0:1789760690.623696:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495692160 00000100:00100000:2.0:1789760690.623708:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495692160 00000100:00100000:2.0:1789760690.623774:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 674796198182 time 674868709343 state 0x0 ref 1 00000100:00100000:2.0:1789760690.623790:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 674868969759 time 674868709343 state 0x1 ref 1 00000100:00100000:2.0:1789760690.623821:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 674868969759 time 674944537887 state 0x0 ref 1 00000100:00100000:2.0:1789760690.623833:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495692160:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760690.623879:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19701760, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0x9807 00010000:00010000:2.0:1789760690.623889:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354544b52 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.623904:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=674s v=(0 0) 00010000:00010000:2.0:1789760690.623908:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760690.623914:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354544b52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.623928:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354544b52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760690.735501:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760690.794866:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495692416 00000100:00100000:2.0:1789760690.794876:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495692416 00000100:00100000:2.0:1789760690.794929:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 674944798303 time 675056430921 state 0x0 ref 1 00000100:00100000:2.0:1789760690.794945:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 675056691337 time 675056430921 state 0x1 ref 1 00000100:00100000:2.0:1789760690.794979:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 675056691337 time 675115694914 state 0x0 ref 1 00000100:00100000:2.0:1789760690.794992:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495692416:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760690.795040:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19709952, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0x9807 00010000:00010000:2.0:1789760690.795050:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354544b60 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.795068:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=675s v=(0 0) 00010000:00010000:2.0:1789760690.795072:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760690.795079:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354544b60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.795093:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354544b60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.043557:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789760691.056139:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789760691.118895:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19718144 -> 19722240 00010000:00010000:2.0:1789760691.118911:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354544b6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.118925:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354544b6e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.118937:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354544b6e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.118945:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760691.118949:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354544b6e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.118958:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354544b6e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.118978:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354544b6e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.118986:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354544b6e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760691.119082:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495692672:12345-192.168.203.9@tcp:4:dd.0 Request processed in 62963us (63377us total) trans 4294972131 rc 0/0 00000100:00100000:2.0:1789760691.119093:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4843 00000100:00100000:2.0:1789760691.119103:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 675224329087 time 675376821620 state 0x2 ref 0 00000100:00100000:2.0:1789760691.122683:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495692800 00000100:00100000:2.0:1789760691.122692:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495692800 00000100:00100000:2.0:1789760691.122805:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 675224329087 time 675376821620 state 0x0 ref 1 00000100:00100000:2.0:1789760691.122821:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 675377082036 time 675376821620 state 0x1 ref 1 00000100:00100000:2.0:1789760691.122856:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 675377082036 time 675443572346 state 0x0 ref 1 00000100:00100000:2.0:1789760691.122868:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495692800:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760691.122916:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19722240, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0x9807 00010000:00010000:2.0:1789760691.122926:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354544b75 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.122943:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=675s v=(0 0) 00010000:00010000:2.0:1789760691.122948:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760691.122955:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354544b75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.122969:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354544b75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760691.291015:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495693056 00000100:00100000:2.0:1789760691.291026:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495693056 00000100:00100000:2.0:1789760691.291081:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 675443832762 time 675493832166 state 0x0 ref 1 00000100:00100000:2.0:1789760691.291097:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 675494092582 time 675493832166 state 0x1 ref 1 00000100:00100000:2.0:1789760691.291134:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 675494092582 time 675611849626 state 0x0 ref 1 00000100:00100000:2.0:1789760691.291146:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495693056:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760691.291195:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19730432, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0x9807 00010000:00010000:2.0:1789760691.291205:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544b83 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.291221:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=675s v=(0 0) 00010000:00010000:2.0:1789760691.291226:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760691.291232:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544b83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.291246:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544b83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760691.369972:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972134 is committed 00002000:00010000:2.0:1789760691.370041:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19730432 -> 19734528 00010000:00010000:2.0:1789760691.370055:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544b83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.370069:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544b83 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.370078:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544b83 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.370091:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760691.370097:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544b83 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.370105:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544b83 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.370125:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544b83 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.370132:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544b83 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760691.370681:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495693056:12345-192.168.203.9@tcp:4:dd.0 Request processed in 79530us (79772us total) trans 4294972134 rc 0/0 00000100:00100000:2.0:1789760691.370692:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4846 00000100:00100000:2.0:1789760691.370702:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 675494092582 time 675611849626 state 0x2 ref 0 00000001:00080000:2.0:1789760691.452781:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972135 is committed 00000100:00100000:2.0:1789760691.456902:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760691.486515:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9290404a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3318:1876699520296960:0@lo:400:kworker.0 00000100:00100000:2.0:1789760691.486583:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760691.486642:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9290404a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3318:1876699520297472:0@lo:400:kworker.0 00000100:00100000:2.0:1789760691.486670:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760691.486700:0:3318:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1789760691.486706:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9290404a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:59c4e264-9b75-482b-8813-3d1dc124f384:3318:1876699520296832:0@lo:400:kworker.0 00000100:00100000:2.0:1789760691.486734:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760691.486754:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9290404a6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3318:1876699520297344:0@lo:400:kworker.0 00000100:00100000:2.0:1789760691.486774:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760691.486800:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9290404a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520297216:0@lo:400:kworker.0 00000100:00100000:2.0:1789760691.486822:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760691.486840:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789760691.486845:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9290404a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699520297088:0@lo:400:kworker.0 00000100:00100000:2.0:1789760691.486867:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760691.486894:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520296960 00000100:00100000:2.0:1789760691.486901:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520296960 00000100:00100000:2.0:1789760691.486940:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520297344 00000100:00100000:2.0:1789760691.486943:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520297344 00000100:00100000:2.0:1789760691.486968:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 432 00000100:00100000:2.0:1789760691.486976:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3318:x1876699520296960:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760691.487025:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3318:x1876699520296960:12345-0@lo:400:kworker.0 Request processed in 49us (441us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760691.487033:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 432 00000100:00100000:2.0:1789760691.487047:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520297088 00000100:00100000:2.0:1789760691.487051:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520297088 00000100:00100000:2.0:1789760691.487066:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 433 00000100:00100000:2.0:1789760691.487072:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04928000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1876699520297344:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760691.487100:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04928000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1876699520297344:12345-0@lo:400:kworker.0 Request processed in 28us (324us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760691.487106:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 433 00000100:00100000:2.0:1789760691.487116:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 434 00000100:00100000:2.0:1789760691.487123:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699520297088:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760691.487151:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699520297088:12345-0@lo:400:kworker.0 Request processed in 28us (283us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760691.487157:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 434 00000100:00100000:2.0:1789760691.487188:0:10490:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520297472 00000100:00100000:2.0:1789760691.487191:0:10490:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520297472 00000100:00100000:2.0:1789760691.487216:0:10490:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520297216 00000100:00100000:2.0:1789760691.487219:0:10490:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520297216 00000100:00100000:2.0:1789760691.487239:0:10490:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 204 00000100:00100000:2.0:1789760691.487244:0:10490:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3318:x1876699520297472:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760691.487275:0:10490:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3318:x1876699520297472:12345-0@lo:400:kworker.0 Request processed in 31us (605us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760691.487309:0:10490:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 204 00000100:00100000:2.0:1789760691.487321:0:10490:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 205 00000100:00100000:2.0:1789760691.487327:0:10490:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04929c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520297216:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760691.487354:0:10490:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04929c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520297216:12345-0@lo:400:kworker.0 Request processed in 28us (531us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760691.487359:0:10490:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 205 00000100:00100000:2.0:1789760691.487374:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520296832 00000100:00100000:2.0:1789760691.487377:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520296832 00000100:00100000:2.0:1789760691.487398:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 381 00000100:00100000:2.0:1789760691.487403:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290062f4850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:59c4e264-9b75-482b-8813-3d1dc124f384+19:3318:x1876699520296832:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760691.487438:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290062f4850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:59c4e264-9b75-482b-8813-3d1dc124f384+19:3318:x1876699520296832:12345-0@lo:400:kworker.0 Request processed in 35us (704us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760691.487444:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 381 00000100:00080000:2.0:1789760691.487478:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789760691.487482:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760691.487484:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760691.487519:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9290404a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699520297088:0@lo:400:kworker.0 00000100:00100000:2.0:1789760691.487527:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9290404a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520297216:0@lo:400:kworker.0 00000100:00100000:2.0:1789760691.487552:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9290404a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:59c4e264-9b75-482b-8813-3d1dc124f384:3318:1876699520296832:0@lo:400:kworker.0 00000100:00100000:2.0:1789760691.487560:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9290404a6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3318:1876699520297344:0@lo:400:kworker.0 00000100:00100000:2.0:1789760691.487569:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9290404a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3318:1876699520296960:0@lo:400:kworker.0 00000100:00100000:2.0:1789760691.487589:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9290404a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3318:1876699520297472:0@lo:400:kworker.0 00002000:00010000:2.0:1789760691.517727:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19738624 -> 19742720 00010000:00010000:2.0:1789760691.517800:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354544b91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.517813:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354544b91 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.517823:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354544b91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.517830:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760691.517836:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354544b91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.517845:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354544b91 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.517869:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354544b91 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.517878:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354544b91 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760691.517963:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495693312:12345-192.168.203.9@tcp:4:dd.0 Request processed in 60870us (61064us total) trans 4294972136 rc 0/0 00000100:00100000:2.0:1789760691.517973:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4848 00000100:00100000:2.0:1789760691.517982:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 675696064215 time 675777795284 state 0x2 ref 0 00000100:00100000:2.0:1789760691.532842:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495693440 00000100:00100000:2.0:1789760691.532852:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495693440 00000100:00100000:2.0:1789760691.532913:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 675696064215 time 675777795284 state 0x0 ref 1 00000100:00100000:2.0:1789760691.532928:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 675778055700 time 675777795284 state 0x1 ref 1 00000100:00100000:2.0:1789760691.532959:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 675778055700 time 675853676158 state 0x0 ref 1 00000100:00100000:2.0:1789760691.532971:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495693440:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760691.533021:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19742720, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0x9807 00010000:00010000:2.0:1789760691.533031:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544b98 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.533049:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=675s v=(0 0) 00010000:00010000:2.0:1789760691.533054:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760691.533061:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544b98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.533074:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544b98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760691.696874:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495693696 00000100:00100000:2.0:1789760691.696884:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495693696 00000100:00100000:2.0:1789760691.697175:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 675853936574 time 675922948646 state 0x0 ref 1 00000100:00100000:2.0:1789760691.697191:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 675923209062 time 675922948646 state 0x1 ref 1 00000100:00100000:2.0:1789760691.697220:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 675923209062 time 676017936964 state 0x0 ref 1 00000100:00100000:2.0:1789760691.697232:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495693696:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760691.697277:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19750912, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0x9807 00010000:00010000:2.0:1789760691.697317:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544ba6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.697333:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=676s v=(0 0) 00010000:00010000:2.0:1789760691.697338:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760691.697344:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544ba6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.697358:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544ba6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760691.827165:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495694080 00000100:00100000:2.0:1789760691.827174:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495694080 00000100:00100000:2.0:1789760691.827230:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676075938547 time 676120684110 state 0x0 ref 1 00000100:00100000:2.0:1789760691.827247:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676120944526 time 676120684110 state 0x1 ref 1 00000100:00100000:2.0:1789760691.827275:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676120944526 time 676147991229 state 0x0 ref 1 00000100:00100000:2.0:1789760691.827314:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495694080:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760691.827368:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19763200, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0x9807 00010000:00010000:2.0:1789760691.827377:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544bbb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.827397:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=676s v=(0 0) 00010000:00010000:2.0:1789760691.827402:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760691.827409:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544bbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.827433:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544bbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760691.989717:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00002000:00010000:2.0:1789760692.036193:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19775488 -> 19779584 00010000:00010000:2.0:1789760692.036209:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354544bd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.036228:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354544bd0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.036237:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354544bd0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.036245:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760692.036249:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354544bd0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.036259:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354544bd0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.036317:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354544bd0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.036327:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354544bd0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760692.036427:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495694464:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45991us (46715us total) trans 4294972145 rc 0/0 00000100:00100000:2.0:1789760692.036437:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4857 00000100:00100000:2.0:1789760692.036448:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676263670523 time 676311129695 state 0x2 ref 0 00000100:00100000:2.0:1789760692.085733:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676311390111 time 676363841351 state 0x0 ref 1 00000100:00100000:2.0:1789760692.085751:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676364101767 time 676363841351 state 0x1 ref 1 00000100:00100000:2.0:1789760692.085783:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676364101767 time 676406499215 state 0x0 ref 1 00000100:00100000:2.0:1789760692.085796:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495694720:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760692.085844:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19783680, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0x9807 00010000:00010000:2.0:1789760692.085854:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544bde lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.085869:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=676s v=(0 0) 00010000:00010000:2.0:1789760692.085874:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760692.085881:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544bde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.085896:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544bde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760692.138646:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972147 is committed 00002000:00010000:2.0:1789760692.141486:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19783680 -> 19787776 00010000:00010000:2.0:1789760692.141577:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544bde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.141591:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544bde lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.141601:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544bde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.141610:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760692.141614:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544bde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.141623:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544bde lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.141645:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544bde lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.141652:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544bde lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760692.141884:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495694720:12345-192.168.203.9@tcp:4:dd.0 Request processed in 56085us (56924us total) trans 4294972147 rc 0/0 00000100:00100000:2.0:1789760692.141894:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4859 00000100:00100000:2.0:1789760692.141904:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676364101767 time 676406499215 state 0x2 ref 0 00000100:00100000:2.0:1789760692.151445:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495694848 00000100:00100000:2.0:1789760692.151455:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495694848 00000100:00100000:2.0:1789760692.151518:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676364101767 time 676406499215 state 0x0 ref 1 00000100:00100000:2.0:1789760692.151533:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676406759631 time 676406499215 state 0x1 ref 1 00000100:00100000:2.0:1789760692.151555:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676406759631 time 676472272217 state 0x0 ref 1 00000100:00100000:2.0:1789760692.151571:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495694848:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760692.151617:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19787776, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0x9807 00010000:00010000:2.0:1789760692.151626:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544be5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.151643:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=676s v=(0 0) 00010000:00010000:2.0:1789760692.151647:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760692.151653:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544be5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.151666:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544be5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760692.242010:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972148 is committed 00000100:00100000:2.0:1789760692.287680:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495695104 00000100:00100000:2.0:1789760692.287702:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495695104 00000100:00100000:2.0:1789760692.287780:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676472532633 time 676572797329 state 0x0 ref 1 00000100:00100000:2.0:1789760692.287799:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676573057745 time 676572797329 state 0x1 ref 1 00000100:00100000:2.0:1789760692.287835:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676573057745 time 676608551339 state 0x0 ref 1 00000100:00100000:2.0:1789760692.287849:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04290700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495695104:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760692.287915:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19795968, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0x9807 00010000:00010000:2.0:1789760692.287925:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544bf3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.287947:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=676s v=(0 0) 00010000:00010000:2.0:1789760692.287951:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760692.287960:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544bf3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.287976:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544bf3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760692.327955:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19795968 -> 19800064 00010000:00010000:2.0:1789760692.327974:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544bf3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.327987:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544bf3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.327996:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544bf3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.328004:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760692.328008:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544bf3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.328018:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544bf3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.328037:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544bf3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.328045:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544bf3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760692.328129:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04290700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495695104:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40276us (40574us total) trans 4294972150 rc 0/0 00000100:00100000:2.0:1789760692.328139:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4862 00000100:00100000:2.0:1789760692.328150:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676573057745 time 676608551339 state 0x2 ref 0 00000001:00080000:2.0:1789760692.328190:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972150 is committed 00000100:00100000:2.0:1789760692.332878:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495695232 00000100:00100000:2.0:1789760692.332889:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495695232 00000100:00100000:2.0:1789760692.332951:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676573057745 time 676608551339 state 0x0 ref 1 00000100:00100000:2.0:1789760692.332967:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676608811755 time 676608551339 state 0x1 ref 1 00000100:00100000:2.0:1789760692.332999:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676608811755 time 676653716282 state 0x0 ref 1 00000100:00100000:2.0:1789760692.333012:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495695232:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760692.333065:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19800064, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0x9807 00010000:00010000:2.0:1789760692.333075:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544bfa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.333093:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=676s v=(0 0) 00010000:00010000:2.0:1789760692.333097:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760692.333104:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544bfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.333119:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544bfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760692.362056:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19800064 -> 19804160 00010000:00010000:2.0:1789760692.362072:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544bfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.362086:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544bfa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.362097:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544bfa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.362105:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760692.362110:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544bfa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.362119:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544bfa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.362138:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544bfa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.362147:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544bfa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760692.362256:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495695232:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29240us (29687us total) trans 4294972151 rc 0/0 00000100:00100000:2.0:1789760692.362268:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4863 00000100:00100000:2.0:1789760692.362313:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676608811755 time 676653716282 state 0x2 ref 0 00000100:00100000:2.0:1789760692.409101:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495695488 00000100:00100000:2.0:1789760692.409111:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495695488 00000100:00100000:2.0:1789760692.409165:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676653976698 time 676688979461 state 0x0 ref 1 00000100:00100000:2.0:1789760692.409181:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676689239877 time 676688979461 state 0x1 ref 1 00000100:00100000:2.0:1789760692.409210:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676689239877 time 676729927977 state 0x0 ref 1 00000100:00100000:2.0:1789760692.409222:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495695488:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760692.409266:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19808256, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0x9807 00010000:00010000:2.0:1789760692.409274:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c08 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.409327:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=676s v=(0 0) 00010000:00010000:2.0:1789760692.409331:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760692.409337:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.409350:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760692.483364:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19808256 -> 19812352 00010000:00010000:2.0:1789760692.483380:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.483393:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c08 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.483427:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c08 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.483434:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760692.483439:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c08 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.483449:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c08 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.483476:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c08 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.483484:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c08 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760692.483589:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495695488:12345-192.168.203.9@tcp:4:dd.0 Request processed in 74356us (74580us total) trans 4294972153 rc 0/0 00000100:00100000:2.0:1789760692.483603:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4865 00000100:00100000:2.0:1789760692.483612:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676689239877 time 676729927977 state 0x2 ref 0 00000100:00100000:2.0:1789760692.489856:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495695616 00000100:00100000:2.0:1789760692.489866:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495695616 00000100:00100000:2.0:1789760692.489920:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676689239877 time 676729927977 state 0x0 ref 1 00000100:00100000:2.0:1789760692.489935:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676730188393 time 676729927977 state 0x1 ref 1 00000100:00100000:2.0:1789760692.489965:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676730188393 time 676810681437 state 0x0 ref 1 00000100:00100000:2.0:1789760692.489977:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04291180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495695616:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760692.490025:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19812352, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0x9807 00010000:00010000:2.0:1789760692.490034:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544c0f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.490051:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=676s v=(0 0) 00010000:00010000:2.0:1789760692.490056:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760692.490181:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544c0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.490196:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544c0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760692.556405:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972154 is committed 00000001:00080000:2.0:1789760692.628370:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972155 is committed 00000100:00100000:2.0:1789760692.635754:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495695872 00000100:00100000:2.0:1789760692.635764:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495695872 00000100:00100000:2.0:1789760692.635820:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676810941853 time 676883232953 state 0x0 ref 1 00000100:00100000:2.0:1789760692.635835:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676883493369 time 676883232953 state 0x1 ref 1 00000100:00100000:2.0:1789760692.640226:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676883493369 time 676960930328 state 0x0 ref 1 00002000:00010000:2.0:1789760692.745413:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19824640 -> 19828736 00010000:00010000:2.0:1789760692.745433:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354544c24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.745446:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354544c24 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.745456:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354544c24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.745463:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760692.745469:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354544c24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.745479:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354544c24 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.745498:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354544c24 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.745510:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354544c24 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760692.745602:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495696000:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51873us (52200us total) trans 4294972157 rc 0/0 00000100:00100000:2.0:1789760692.745613:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4869 00000100:00100000:2.0:1789760692.745623:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 676961190744 time 677014430577 state 0x2 ref 0 00002000:00010000:2.0:1789760692.781478:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19828736 -> 19832832 00010000:00010000:2.0:1789760692.781494:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354544c2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.781507:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354544c2b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.781517:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354544c2b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.781524:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760692.781530:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354544c2b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.781539:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354544c2b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.781564:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354544c2b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.781572:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354544c2b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760692.781663:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290363a2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495696512:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32057us (32588us total) trans 4294972158 rc 0/0 00000100:00100000:2.0:1789760692.781672:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4870 00000100:00100000:2.0:1789760692.781682:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677014690993 time 677070308443 state 0x2 ref 0 00000100:00100000:2.0:1789760692.785673:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495696640 00000100:00100000:2.0:1789760692.785683:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495696640 00000100:00100000:2.0:1789760692.785736:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677014690993 time 677070308443 state 0x0 ref 1 00000100:00100000:2.0:1789760692.785751:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677070568859 time 677070308443 state 0x1 ref 1 00000100:00100000:2.0:1789760692.785814:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677070568859 time 677106530826 state 0x0 ref 1 00000100:00100000:2.0:1789760692.785826:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495696640:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760692.785872:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19832832, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0x9807 00010000:00010000:2.0:1789760692.785881:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c32 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.785897:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=677s v=(0 0) 00010000:00010000:2.0:1789760692.785902:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760692.785908:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.785920:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760692.807981:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19832832 -> 19836928 00010000:00010000:2.0:1789760692.807997:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.808011:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c32 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.808021:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.808029:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760692.808034:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.808045:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c32 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.808064:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c32 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.808077:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c32 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760692.808202:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495696640:12345-192.168.203.9@tcp:4:dd.0 Request processed in 22373us (22690us total) trans 4294972159 rc 0/0 00000100:00100000:2.0:1789760692.808213:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4871 00000100:00100000:2.0:1789760692.808224:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677070568859 time 677106530826 state 0x2 ref 0 00000100:00100000:2.0:1789760692.814455:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495696768 00000100:00100000:2.0:1789760692.814464:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495696768 00000100:00100000:2.0:1789760692.814532:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677070568859 time 677106530826 state 0x0 ref 1 00000100:00100000:2.0:1789760692.814547:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677106791242 time 677106530826 state 0x1 ref 1 00000100:00100000:2.0:1789760692.814577:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677106791242 time 677135293751 state 0x0 ref 1 00000100:00100000:2.0:1789760692.814588:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03095500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495696768:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760692.815258:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19836928, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0x9807 00010000:00010000:2.0:1789760692.815270:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c39 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.815308:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=677s v=(0 0) 00010000:00010000:2.0:1789760692.815313:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760692.815319:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.815338:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760692.835463:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19836928 -> 19841024 00010000:00010000:2.0:1789760692.835478:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.835490:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c39 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.835498:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.835505:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760692.835510:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.835518:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c39 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.835534:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c39 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.835541:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c39 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760692.835626:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495696768:12345-192.168.203.9@tcp:4:dd.0 Request processed in 21034us (21472us total) trans 4294972160 rc 0/0 00000100:00100000:2.0:1789760692.835637:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4872 00000100:00100000:2.0:1789760692.835647:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677106791242 time 677135293751 state 0x2 ref 0 00000100:00100000:2.0:1789760692.841146:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495696896 00000100:00100000:2.0:1789760692.841156:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495696896 00000100:00100000:2.0:1789760692.841211:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677106791242 time 677135293751 state 0x0 ref 1 00000100:00100000:2.0:1789760692.841226:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677135554167 time 677135293751 state 0x1 ref 1 00000100:00100000:2.0:1789760692.841260:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677135554167 time 677161975823 state 0x0 ref 1 00000100:00100000:2.0:1789760692.841324:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495696896:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760692.841373:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19841024, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0x9807 00010000:00010000:2.0:1789760692.841382:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c40 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.841398:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=677s v=(0 0) 00010000:00010000:2.0:1789760692.841402:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760692.841423:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.841438:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760692.865502:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19841024 -> 19845120 00010000:00010000:2.0:1789760692.865519:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.865532:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c40 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.865540:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.865547:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760692.865552:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.865561:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c40 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.865591:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c40 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.865600:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544c40 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760692.865742:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495696896:12345-192.168.203.9@tcp:4:dd.0 Request processed in 24458us (24644us total) trans 4294972161 rc 0/0 00000100:00100000:2.0:1789760692.865757:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4873 00000100:00100000:2.0:1789760692.865767:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677135554167 time 677161975823 state 0x2 ref 0 00000100:00100000:2.0:1789760692.871934:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495697024 00000100:00100000:2.0:1789760692.871945:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495697024 00000100:00100000:2.0:1789760692.871995:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677135554167 time 677161975823 state 0x0 ref 1 00000100:00100000:2.0:1789760692.872013:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677162236239 time 677161975823 state 0x1 ref 1 00000100:00100000:2.0:1789760692.872040:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677162236239 time 677192757014 state 0x0 ref 1 00000100:00100000:2.0:1789760692.872052:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495697024:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760692.872096:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19845120, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0x9807 00010000:00010000:2.0:1789760692.872105:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544c47 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.872121:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=677s v=(0 0) 00010000:00010000:2.0:1789760692.872125:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760692.872131:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544c47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.872154:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544c47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760692.910374:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972162 is committed 00000100:00100000:2.0:1789760692.914194:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495697152 00000100:00100000:2.0:1789760692.914209:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495697152 00000100:00100000:2.0:1789760692.914252:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677162236239 time 677192757014 state 0x0 ref 1 00000100:00100000:2.0:1789760692.914264:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677193017430 time 677192757014 state 0x1 ref 1 00000100:00100000:2.0:1789760692.914460:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677193017430 time 677235175111 state 0x0 ref 1 00000100:00100000:2.0:1789760692.914473:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03097b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495697152:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760692.914517:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19849216, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0x9807 00010000:00010000:2.0:1789760692.914523:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544c4e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.914536:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=677s v=(0 0) 00010000:00010000:2.0:1789760692.914539:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760692.914543:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544c4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.914553:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544c4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760692.948162:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19849216 -> 19853312 00010000:00010000:2.0:1789760692.948184:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544c4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.948209:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544c4e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.948226:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544c4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.948234:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760692.948239:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544c4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.948248:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544c4e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.948268:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544c4e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.948276:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544c4e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760692.948400:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03097b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495697152:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33913us (34258us total) trans 4294972163 rc 0/0 00000100:00100000:2.0:1789760692.948420:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4875 00000100:00100000:2.0:1789760692.948430:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677193017430 time 677235175111 state 0x2 ref 0 00000001:00080000:2.0:1789760692.948470:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972163 is committed 00000100:00100000:2.0:1789760692.955016:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760693.025888:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00080000:2.0:1789760693.026320:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760693.168053:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760693.225503:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760693.289445:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00002000:00010000:2.0:1789760693.290746:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19869696, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0x9807 00010000:00010000:2.0:1789760693.290757:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354544c71 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.290777:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=677s v=(0 0) 00010000:00010000:2.0:1789760693.290782:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760693.290788:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354544c71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.290801:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354544c71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760693.418416:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972168 is committed 00000100:00100000:2.0:1789760693.428514:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760693.439157:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495697920 00000100:00100000:1.0:1789760693.439172:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495697920 00000100:00100000:1.0:1789760693.439238:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677546684984 time 677611170364 state 0x0 ref 1 00000100:00100000:1.0:1789760693.439254:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677611430780 time 677611170364 state 0x1 ref 1 00000100:00100000:1.0:1789760693.439311:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677611430780 time 677760027662 state 0x0 ref 1 00000100:00100000:1.0:1789760693.439325:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495697920:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760693.439373:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19873792, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0x9807 00010000:00010000:1.0:1789760693.439383:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354544c78 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.439409:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=677s v=(0 0) 00010000:00010000:1.0:1789760693.439414:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760693.439421:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354544c78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.439435:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354544c78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760693.609509:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19873792 -> 19877888 00010000:00010000:1.0:1789760693.609524:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354544c78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.609537:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354544c78 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.609551:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354544c78 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.609558:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760693.609564:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354544c78 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.609573:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354544c78 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.609592:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354544c78 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.609599:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354544c78 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760693.609967:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495697920:12345-192.168.203.9@tcp:4:dd.0 Request processed in 170640us (181459us total) trans 4294972169 rc 0/0 00000100:00100000:1.0:1789760693.609977:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4881 00000100:00100000:1.0:1789760693.609986:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677611430780 time 677760027662 state 0x2 ref 0 00000100:00100000:2.0:1789760693.638256:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760693.638832:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495698048 00000100:00100000:1.0:1789760693.638843:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495698048 00000100:00100000:1.0:1789760693.638977:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677611430780 time 677760027662 state 0x0 ref 1 00000100:00100000:1.0:1789760693.639086:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677760288078 time 677760027662 state 0x1 ref 1 00000100:00100000:1.0:1789760693.639120:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677760288078 time 677959836539 state 0x0 ref 1 00000100:00100000:1.0:1789760693.639133:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495698048:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760693.639181:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19877888, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0x9807 00010000:00010000:1.0:1789760693.639190:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354544c7f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.639206:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=677s v=(0 0) 00010000:00010000:1.0:1789760693.639210:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760693.639217:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354544c7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.639231:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354544c7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760693.765460:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495698176 00000100:00100000:1.0:1789760693.765470:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495698176 00000100:00100000:1.0:1789760693.765529:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677760288078 time 677959836539 state 0x0 ref 1 00000100:00100000:1.0:1789760693.765545:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677960096955 time 677959836539 state 0x1 ref 1 00000100:00100000:1.0:1789760693.765571:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677960096955 time 678086287466 state 0x0 ref 1 00000100:00100000:1.0:1789760693.765584:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495698176:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760693.765635:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19881984, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0x9807 00010000:00010000:1.0:1789760693.765646:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354544c86 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.765664:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=678s v=(0 0) 00010000:00010000:1.0:1789760693.765668:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760693.765675:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354544c86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.765688:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354544c86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760693.766011:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972170 is committed 00000001:00080000:2.0:1789760693.799110:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972171 is committed 00002000:00010000:2.0:1789760693.799189:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19881984 -> 19886080 00010000:00010000:2.0:1789760693.799204:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354544c86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.799216:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354544c86 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.799226:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354544c86 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.799233:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760693.799238:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354544c86 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.799247:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354544c86 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.799267:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354544c86 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.799275:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354544c86 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760693.799416:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495698176:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33830us (34094us total) trans 4294972171 rc 0/0 00000100:00100000:2.0:1789760693.799427:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4883 00000100:00100000:2.0:1789760693.799438:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677960096955 time 678086287466 state 0x2 ref 0 00000100:00100000:1.0:1789760693.808828:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495698304 00000100:00100000:1.0:1789760693.808839:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495698304 00000100:00100000:1.0:1789760693.808901:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 677960096955 time 678086287466 state 0x0 ref 1 00000100:00100000:1.0:1789760693.808917:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678086547882 time 678086287466 state 0x1 ref 1 00000100:00100000:1.0:1789760693.808946:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678086547882 time 678129663197 state 0x0 ref 1 00000100:00100000:1.0:1789760693.808958:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495698304:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760693.809006:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19886080, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0x9807 00010000:00010000:1.0:1789760693.809016:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354544c8d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.809032:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=678s v=(0 0) 00010000:00010000:1.0:1789760693.809037:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760693.809045:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354544c8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.809059:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354544c8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760693.871543:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19886080 -> 19890176 00010000:00010000:1.0:1789760693.871559:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354544c8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.871573:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354544c8d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.871582:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354544c8d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.871589:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760693.871594:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354544c8d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.871603:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354544c8d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.871622:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354544c8d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.871629:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354544c8d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760693.873350:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495698304:12345-192.168.203.9@tcp:4:dd.0 Request processed in 64385us (64650us total) trans 4294972172 rc 0/0 00000100:00100000:1.0:1789760693.873364:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4884 00000100:00100000:1.0:1789760693.874989:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678086547882 time 678129663197 state 0x2 ref 0 00000100:00100000:1.0:1789760693.876861:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495698432 00000100:00100000:1.0:1789760693.876874:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495698432 00000100:00100000:1.0:1789760693.876928:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678086547882 time 678129663197 state 0x0 ref 1 00000100:00100000:1.0:1789760693.876948:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678129923613 time 678129663197 state 0x1 ref 1 00000100:00100000:1.0:1789760693.876978:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678129923613 time 678197694778 state 0x0 ref 1 00000100:00100000:1.0:1789760693.876992:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495698432:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760693.877036:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19890176, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0x9807 00010000:00010000:1.0:1789760693.877045:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354544c94 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.877072:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=678s v=(0 0) 00010000:00010000:1.0:1789760693.877077:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760693.877084:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354544c94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.877099:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354544c94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760693.934915:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972173 is committed 00002000:00010000:1.0:1789760693.936887:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19890176 -> 19894272 00010000:00010000:1.0:1789760693.936906:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354544c94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.936919:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354544c94 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.936928:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354544c94 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.936934:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760693.936939:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354544c94 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.936949:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354544c94 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.936968:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354544c94 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.936974:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354544c94 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760693.937403:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495698432:12345-192.168.203.9@tcp:4:dd.0 Request processed in 60399us (61531us total) trans 4294972173 rc 0/0 00000100:00100000:1.0:1789760693.937419:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4885 00000100:00100000:1.0:1789760693.937429:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678129923613 time 678197694778 state 0x2 ref 0 00000100:00100000:1.0:1789760693.946562:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495698560 00000100:00100000:1.0:1789760693.946572:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495698560 00000100:00100000:1.0:1789760693.946633:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678129923613 time 678197694778 state 0x0 ref 1 00000100:00100000:1.0:1789760693.946650:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678197955194 time 678197694778 state 0x1 ref 1 00000100:00100000:1.0:1789760693.946680:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678197955194 time 678267396407 state 0x0 ref 1 00000100:00100000:1.0:1789760693.946692:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495698560:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760693.946747:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19894272, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0x9807 00010000:00010000:1.0:1789760693.946757:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354544c9b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.946772:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=678s v=(0 0) 00010000:00010000:1.0:1789760693.946777:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760693.946783:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354544c9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.946797:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354544c9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760694.051153:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19894272 -> 19898368 00010000:00010000:2.0:1789760694.051172:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354544c9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.051186:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354544c9b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.051203:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354544c9b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.051210:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760694.051215:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354544c9b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.051223:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354544c9b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.051241:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354544c9b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.051248:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354544c9b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760694.051388:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495698560:12345-192.168.203.9@tcp:4:dd.0 Request processed in 104684us (104934us total) trans 4294972174 rc 0/0 00000100:00100000:2.0:1789760694.051408:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4886 00000100:00100000:2.0:1789760694.051417:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678197955194 time 678267396407 state 0x2 ref 0 00000100:00100000:2.0:1789760694.056472:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760694.140866:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678377831796 time 678461582740 state 0x0 ref 1 00000100:00100000:0.0:1789760694.140877:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495698816:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760694.140924:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19902464, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0x9807 00010000:00010000:0.0:1789760694.140933:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354544ca9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.140958:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=678s v=(0 0) 00010000:00010000:0.0:1789760694.140963:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760694.140969:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354544ca9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.140984:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354544ca9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760694.171536:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972176 is committed 00002000:00010000:0.0:1789760694.171953:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19902464 -> 19906560 00010000:00010000:0.0:1789760694.171969:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354544ca9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.171982:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354544ca9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.171992:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354544ca9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.171997:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760694.172001:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354544ca9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.172008:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354544ca9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.172023:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354544ca9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.172028:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354544ca9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760694.172108:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495698816:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31228us (31444us total) trans 4294972176 rc 0/0 00000100:00100000:0.0:1789760694.172116:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4888 00000100:00100000:0.0:1789760694.172123:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678377831796 time 678461582740 state 0x2 ref 0 00000100:00100000:0.0:1789760694.175906:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760694.176058:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495698944 00000100:00100000:2.0:1789760694.176068:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495698944 00000100:00100000:2.0:1789760694.176124:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678377831796 time 678461582740 state 0x0 ref 1 00000100:00100000:2.0:1789760694.176137:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678461843156 time 678461582740 state 0x1 ref 1 00000100:00100000:2.0:1789760694.176167:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678461843156 time 678496883953 state 0x0 ref 1 00000100:00100000:2.0:1789760694.176180:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495698944:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760694.176227:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19906560, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0x9807 00010000:00010000:2.0:1789760694.176235:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544cb0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.176247:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=678s v=(0 0) 00010000:00010000:2.0:1789760694.176250:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760694.176254:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544cb0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.176264:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354544cb0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760694.199766:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972177 is committed 00000100:00100000:2.0:1789760694.202371:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:0.0:1789760694.263017:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972178 is committed 00000100:00100000:2.0:1789760694.268624:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760694.268815:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495699200 00000100:00100000:0.0:1789760694.268825:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495699200 00000100:00100000:0.0:1789760694.268882:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678497144369 time 678523340986 state 0x0 ref 1 00000100:00100000:0.0:1789760694.268896:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678523601402 time 678523340986 state 0x1 ref 1 00000100:00100000:0.0:1789760694.268917:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678523601402 time 678589634908 state 0x0 ref 1 00000100:00100000:0.0:1789760694.268929:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495699200:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760694.268976:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19914752, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0x9807 00010000:00010000:0.0:1789760694.268985:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354544cbe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.269000:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=678s v=(0 0) 00010000:00010000:0.0:1789760694.269005:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760694.269011:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354544cbe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.269024:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354544cbe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760694.341867:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972179 is committed 00002000:00010000:1.0:1789760694.341989:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19914752 -> 19918848 00010000:00010000:1.0:1789760694.342004:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354544cbe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.342018:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354544cbe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.342028:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354544cbe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.342037:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760694.342042:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354544cbe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.342050:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354544cbe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.342069:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354544cbe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.342076:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354544cbe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760694.342164:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495699200:12345-192.168.203.9@tcp:4:dd.0 Request processed in 73232us (73546us total) trans 4294972179 rc 0/0 00000100:00100000:1.0:1789760694.342175:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4891 00000100:00100000:1.0:1789760694.342185:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678523601402 time 678589634908 state 0x2 ref 0 00000100:00100000:0.0:1789760694.345995:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:2.0:1789760694.402082:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972180 is committed 00002000:00010000:0.0:1789760694.402148:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19918848 -> 19922944 00010000:00010000:0.0:1789760694.402163:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544cc5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.402175:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544cc5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.402183:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544cc5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.402191:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760694.402196:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544cc5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.402209:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544cc5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.402228:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544cc5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.402236:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544cc5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760694.402521:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495699328:12345-192.168.203.9@tcp:4:dd.0 Request processed in 56262us (56528us total) trans 4294972180 rc 0/0 00000100:00100000:0.0:1789760694.402532:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4892 00000100:00100000:0.0:1789760694.402541:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678589895324 time 678666958598 state 0x2 ref 0 00000100:00100000:2.0:1789760694.406408:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:1.0:1789760694.486556:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972181 is committed 00002000:00010000:0.0:1789760694.486649:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19922944 -> 19927040 00002000:00010000:0.0:1789760694.486652:0:6714:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 38919 -> 40967 00010000:00010000:0.0:1789760694.486665:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544ccc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.486675:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544ccc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.486681:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544ccc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.486689:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760694.486693:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544ccc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.486700:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544ccc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.486716:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544ccc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.486721:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544ccc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760694.486915:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92901237ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495699456:12345-192.168.203.9@tcp:4:dd.0 Request processed in 80266us (80521us total) trans 4294972181 rc 0/0 00000100:00100000:0.0:1789760694.486923:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4893 00000100:00100000:0.0:1789760694.486931:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678667219014 time 678727352663 state 0x2 ref 0 00000100:00100000:2.0:1789760694.489840:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760694.490078:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495699584 00000100:00100000:1.0:1789760694.490089:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495699584 00000100:00100000:1.0:1789760694.490144:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678667219014 time 678727352663 state 0x0 ref 1 00000100:00100000:1.0:1789760694.490160:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678727613079 time 678727352663 state 0x1 ref 1 00000001:00080000:1.0:1789760694.546847:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972182 is committed 00002000:00010000:0.0:1789760694.547016:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19927040 -> 19931136 00010000:00010000:0.0:1789760694.547032:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544cd3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.547045:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544cd3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.547060:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544cd3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.547067:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760694.547073:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544cd3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.547082:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544cd3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.547104:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544cd3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.547112:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544cd3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760694.547576:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fdc9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495699584:12345-192.168.203.9@tcp:4:dd.0 Request processed in 57091us (57737us total) trans 4294972182 rc 0/0 00000100:00100000:0.0:1789760694.547588:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4894 00000100:00100000:0.0:1789760694.547598:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678727613079 time 678811180589 state 0x2 ref 0 00000100:00100000:2.0:1789760694.550552:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760694.550684:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495699712 00000100:00100000:0.0:1789760694.550695:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495699712 00000100:00100000:0.0:1789760694.550765:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678727613079 time 678811180589 state 0x0 ref 1 00000100:00100000:0.0:1789760694.550781:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678811441005 time 678811180589 state 0x1 ref 1 00000100:00100000:0.0:1789760694.550814:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678811441005 time 678871531166 state 0x0 ref 1 00000100:00100000:0.0:1789760694.550826:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fdcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495699712:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760694.550878:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19931136, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0xa007 00010000:00010000:0.0:1789760694.550888:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544cda lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.550904:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=678s v=(0 0) 00010000:00010000:0.0:1789760694.550908:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760694.550915:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544cda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.550929:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544cda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760694.559347:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760694.559367:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f03097100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520297728:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760694.559852:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760694.559920:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520297728 00000100:00100000:1.0:1789760694.559927:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520297728 00000100:00100000:1.0:1789760694.560036:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 181 00000100:00100000:1.0:1789760694.560048:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03096680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520297728:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760694.560070:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527892480 pending 1703936 free 3838836736 avail 3809476608 00000100:00100000:1.0:1789760694.560113:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03096680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520297728:12345-0@lo:13:osp-pre-0-0.0 Request processed in 68us (261us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760694.560121:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 181 00000100:00100000:1.0:1789760694.560168:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f03097100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520297728:0@lo:13:osp-pre-0-0.0 00000100:00080000:2.0:1789760694.561791:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760694.613201:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972183 is committed 00002000:00010000:0.0:1789760694.613592:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19931136 -> 19935232 00010000:00010000:0.0:1789760694.613608:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544cda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.613623:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544cda lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.613632:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544cda lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.613640:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760694.613646:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544cda lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.613655:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544cda lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.613676:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544cda lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.613685:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544cda lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760694.613790:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fdcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495699712:12345-192.168.203.9@tcp:4:dd.0 Request processed in 62961us (63241us total) trans 4294972183 rc 0/0 00000100:00100000:0.0:1789760694.613801:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4895 00000100:00100000:0.0:1789760694.613811:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678811441005 time 678871531166 state 0x2 ref 0 00000100:00100000:2.0:1789760694.621095:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760694.621191:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495699840 00000100:00100000:0.0:1789760694.621201:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495699840 00000100:00100000:0.0:1789760694.621255:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678811441005 time 678871531166 state 0x0 ref 1 00000100:00100000:0.0:1789760694.621269:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678871791582 time 678871531166 state 0x1 ref 1 00000100:00100000:0.0:1789760694.621319:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678871791582 time 678942036135 state 0x0 ref 1 00000100:00100000:0.0:1789760694.621331:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01c8c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495699840:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760694.621385:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19935232, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0xa007 00010000:00010000:0.0:1789760694.621402:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354544ce1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.621418:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=678s v=(0 0) 00010000:00010000:0.0:1789760694.621422:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760694.621429:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354544ce1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.621442:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354544ce1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760694.664427:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972184 is committed 00002000:00010000:1.0:1789760694.664649:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19935232 -> 19939328 00002000:00010000:1.0:1789760694.664654:0:6714:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760693 -> 1789760694 00002000:00010000:1.0:1789760694.664656:0:6714:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760693 -> 1789760694 00010000:00010000:1.0:1789760694.664670:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354544ce1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.664683:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354544ce1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.664699:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354544ce1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.664706:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760694.664784:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354544ce1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.664795:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354544ce1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.664815:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354544ce1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.664823:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354544ce1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760694.664948:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01c8c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495699840:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43614us (43856us total) trans 4294972184 rc 0/0 00000100:00100000:1.0:1789760694.664958:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4896 00000100:00100000:1.0:1789760694.664967:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678871791582 time 678942036135 state 0x2 ref 0 00000100:00100000:0.0:1789760694.674248:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760694.674554:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495699968 00000100:00100000:1.0:1789760694.674564:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495699968 00000100:00100000:1.0:1789760694.674616:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678871791582 time 678942036135 state 0x0 ref 1 00000100:00100000:1.0:1789760694.674630:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678942296551 time 678942036135 state 0x1 ref 1 00000100:00100000:1.0:1789760694.674710:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678942296551 time 678995425520 state 0x0 ref 1 00000100:00100000:1.0:1789760694.674723:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495699968:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760694.674781:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19939328, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:1.0:1789760694.674791:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354544ce8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.674806:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=678s v=(0 0) 00010000:00010000:1.0:1789760694.674810:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760694.674817:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354544ce8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.674830:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354544ce8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760694.716832:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972185 is committed 00002000:00010000:0.0:1789760694.717010:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19939328 -> 19943424 00010000:00010000:0.0:1789760694.717025:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354544ce8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.717037:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354544ce8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.717046:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354544ce8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.717054:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760694.717058:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354544ce8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.717068:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354544ce8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.717090:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354544ce8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.717097:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354544ce8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760694.717198:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495699968:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42472us (43019us total) trans 4294972185 rc 0/0 00000100:00100000:0.0:1789760694.717208:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4897 00000100:00100000:0.0:1789760694.717217:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678942296551 time 678995425520 state 0x2 ref 0 00000100:00100000:2.0:1789760694.725952:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760694.726228:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495700096 00000100:00100000:0.0:1789760694.726238:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495700096 00000100:00100000:0.0:1789760694.726344:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678942296551 time 678995425520 state 0x0 ref 1 00000100:00100000:0.0:1789760694.726361:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678995685936 time 678995425520 state 0x1 ref 1 00000100:00100000:0.0:1789760694.726674:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678995685936 time 679047388952 state 0x0 ref 1 00000100:00100000:0.0:1789760694.726689:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01c8d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495700096:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760694.726752:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19943424, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:0.0:1789760694.726762:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544cef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.726777:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=679s v=(0 0) 00010000:00010000:0.0:1789760694.726782:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760694.726788:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544cef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.726800:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544cef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760694.775026:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19943424 -> 19947520 00010000:00010000:0.0:1789760694.775047:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544cef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.775060:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544cef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.775068:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544cef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.775076:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760694.775081:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544cef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.775090:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544cef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.775109:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544cef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.775117:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544cef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760694.775218:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01c8d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495700096:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48528us (49271us total) trans 4294972186 rc 0/0 00000100:00100000:0.0:1789760694.775230:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4898 00000100:00100000:0.0:1789760694.775241:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678995685936 time 679047388952 state 0x2 ref 0 00000100:00100000:1.0:1789760694.778939:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495700224 00000100:00100000:1.0:1789760694.778949:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495700224 00000100:00100000:1.0:1789760694.779002:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 678995685936 time 679047388952 state 0x0 ref 1 00000100:00100000:1.0:1789760694.779018:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679047649368 time 679047388952 state 0x1 ref 1 00000100:00100000:1.0:1789760694.779049:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679047649368 time 679099765649 state 0x0 ref 1 00000100:00100000:1.0:1789760694.779062:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495700224:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760694.779110:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19947520, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:1.0:1789760694.779120:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354544cf6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.779136:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=679s v=(0 0) 00010000:00010000:1.0:1789760694.779141:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760694.779147:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354544cf6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.779161:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354544cf6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760694.822875:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19947520 -> 19951616 00010000:00010000:0.0:1789760694.822889:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354544cf6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.822902:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354544cf6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.822911:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354544cf6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.822917:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760694.822922:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354544cf6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.822931:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354544cf6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.822949:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354544cf6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.822961:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354544cf6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760694.823072:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495700224:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44009us (44283us total) trans 4294972187 rc 0/0 00000100:00100000:0.0:1789760694.823081:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4899 00000100:00100000:0.0:1789760694.823091:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679047649368 time 679099765649 state 0x2 ref 0 00000100:00100000:0.0:1789760694.826762:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760694.826839:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495700352 00000100:00100000:2.0:1789760694.826848:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495700352 00000100:00100000:2.0:1789760694.826899:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679047649368 time 679099765649 state 0x0 ref 1 00000100:00100000:2.0:1789760694.826915:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679100026065 time 679099765649 state 0x1 ref 1 00000100:00100000:2.0:1789760694.826948:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679100026065 time 679147664915 state 0x0 ref 1 00000100:00100000:2.0:1789760694.826960:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04930000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495700352:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760694.827005:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19951616, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:2.0:1789760694.827015:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544cfd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.827031:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=679s v=(0 0) 00010000:00010000:2.0:1789760694.827036:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760694.827042:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544cfd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.827056:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544cfd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760694.866155:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495700480 00000100:00100000:0.0:1789760694.866166:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495700480 00000100:00100000:0.0:1789760694.866327:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679100026065 time 679147664915 state 0x0 ref 1 00000100:00100000:0.0:1789760694.866342:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679147925331 time 679147664915 state 0x1 ref 1 00000100:00100000:0.0:1789760694.866364:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679147925331 time 679187081022 state 0x0 ref 1 00000100:00100000:0.0:1789760694.866385:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495700480:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760694.866439:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19955712, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:0.0:1789760694.866449:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d04 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.866559:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=679s v=(0 0) 00010000:00010000:0.0:1789760694.866564:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760694.866571:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.866585:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760694.867305:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972188 is committed 00002000:00010000:0.0:1789760694.910102:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19955712 -> 19959808 00010000:00010000:0.0:1789760694.910123:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.910136:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d04 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.910145:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d04 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.910153:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760694.910159:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d04 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.910168:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d04 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.910187:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d04 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.910195:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d04 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760694.910335:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495700480:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43946us (44661us total) trans 4294972189 rc 0/0 00000100:00100000:0.0:1789760694.910345:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4901 00000100:00100000:0.0:1789760694.910356:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679147925331 time 679187081022 state 0x2 ref 0 00000001:00080000:2.0:1789760694.912873:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972189 is committed 00000100:00100000:0.0:1789760694.915396:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760694.915727:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495700608 00000100:00100000:1.0:1789760694.915737:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495700608 00000100:00100000:1.0:1789760694.915788:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679147925331 time 679187081022 state 0x0 ref 1 00000100:00100000:1.0:1789760694.915802:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679187341438 time 679187081022 state 0x1 ref 1 00000100:00100000:1.0:1789760694.915830:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679187341438 time 679236547128 state 0x0 ref 1 00000100:00100000:1.0:1789760694.915842:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04931500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495700608:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760694.915890:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19959808, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:1.0:1789760694.915901:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544d0b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.915917:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=679s v=(0 0) 00010000:00010000:1.0:1789760694.915921:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760694.915928:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544d0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.915942:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544d0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760694.955041:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972190 is committed 00000100:00100000:2.0:1789760694.962202:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495700736 00000100:00100000:2.0:1789760694.962216:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495700736 00000100:00100000:2.0:1789760694.962328:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679187341438 time 679236547128 state 0x0 ref 1 00000100:00100000:2.0:1789760694.962348:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679236807544 time 679236547128 state 0x1 ref 1 00000100:00100000:2.0:1789760694.962383:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679236807544 time 679283099305 state 0x0 ref 1 00000100:00100000:2.0:1789760694.962408:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495700736:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760694.962474:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19963904, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:2.0:1789760694.962486:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544d12 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.962508:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=679s v=(0 0) 00010000:00010000:2.0:1789760694.962513:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760694.962520:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544d12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.962537:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544d12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760695.036141:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972191 is committed 00002000:00010000:0.0:1789760695.036330:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19963904 -> 19968000 00010000:00010000:0.0:1789760695.036346:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544d12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.036360:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544d12 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.036379:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544d12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.036386:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760695.036399:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544d12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.036410:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544d12 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.036429:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544d12 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.036442:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544d12 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760695.036548:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495700736:12345-192.168.203.9@tcp:4:dd.0 Request processed in 74139us (74747us total) trans 4294972191 rc 0/0 00000100:00100000:0.0:1789760695.037241:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4903 00000100:00100000:0.0:1789760695.037251:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679236807544 time 679283099305 state 0x2 ref 0 00000100:00100000:0.0:1789760695.041727:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495700864 00000100:00100000:0.0:1789760695.041739:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495700864 00000100:00100000:0.0:1789760695.041800:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679236807544 time 679283099305 state 0x0 ref 1 00000100:00100000:0.0:1789760695.041815:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679283359721 time 679283099305 state 0x1 ref 1 00000100:00100000:0.0:1789760695.041845:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679283359721 time 679362561628 state 0x0 ref 1 00000100:00100000:0.0:1789760695.041857:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495700864:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760695.041903:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19968000, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:0.0:1789760695.041912:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d19 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.041928:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=679s v=(0 0) 00010000:00010000:0.0:1789760695.041932:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760695.041939:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.041958:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760695.042438:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789760695.098326:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19968000 -> 19972096 00010000:00010000:0.0:1789760695.098342:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.098607:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d19 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.098619:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.098627:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760695.098632:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.098640:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d19 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.098660:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d19 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.098668:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d19 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760695.098832:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495700864:12345-192.168.203.9@tcp:4:dd.0 Request processed in 56972us (57622us total) trans 4294972192 rc 0/0 00000100:00100000:0.0:1789760695.098842:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4904 00000100:00100000:0.0:1789760695.098852:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679283359721 time 679362561628 state 0x2 ref 0 00000100:00100000:0.0:1789760695.103889:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495700992 00000100:00100000:0.0:1789760695.103899:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495700992 00000100:00100000:0.0:1789760695.103952:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679283359721 time 679362561628 state 0x0 ref 1 00000100:00100000:0.0:1789760695.103966:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679362822044 time 679362561628 state 0x1 ref 1 00000100:00100000:0.0:1789760695.103995:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679362822044 time 679424712054 state 0x0 ref 1 00000100:00100000:0.0:1789760695.104009:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495700992:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760695.104056:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19972096, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:0.0:1789760695.104065:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544d20 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.104081:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=679s v=(0 0) 00010000:00010000:0.0:1789760695.104085:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760695.104091:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544d20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.104105:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544d20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.134963:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760695.138219:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19972096 -> 19976192 00010000:00010000:0.0:1789760695.138233:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544d20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.138242:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544d20 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.138248:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544d20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.138253:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760695.138257:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544d20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.138263:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544d20 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.138305:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544d20 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.138311:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544d20 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760695.138727:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495700992:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34712us (34971us total) trans 4294972193 rc 0/0 00000100:00100000:0.0:1789760695.138739:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4905 00000100:00100000:0.0:1789760695.138748:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679362822044 time 679424712054 state 0x2 ref 0 00000100:00100000:0.0:1789760695.143080:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495701120 00000100:00100000:0.0:1789760695.143088:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495701120 00000100:00100000:0.0:1789760695.143141:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679362822044 time 679424712054 state 0x0 ref 1 00000100:00100000:0.0:1789760695.143155:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679424972470 time 679424712054 state 0x1 ref 1 00000100:00100000:0.0:1789760695.143213:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679424972470 time 679463929380 state 0x0 ref 1 00000100:00100000:0.0:1789760695.143222:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495701120:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760695.143265:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19976192, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:0.0:1789760695.143272:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d27 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.143333:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=679s v=(0 0) 00010000:00010000:0.0:1789760695.143337:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760695.143344:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.143373:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760695.184141:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972194 is committed 00002000:00010000:0.0:1789760695.184377:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19976192 -> 19980288 00010000:00010000:0.0:1789760695.184401:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.184415:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d27 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.184425:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d27 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.184432:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760695.184437:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d27 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.184446:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d27 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.184466:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d27 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.184473:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d27 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760695.184563:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495701120:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41336us (41947us total) trans 4294972194 rc 0/0 00000100:00100000:0.0:1789760695.184573:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4906 00000100:00100000:0.0:1789760695.184582:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679424972470 time 679463929380 state 0x2 ref 0 00000100:00100000:0.0:1789760695.189884:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495701248 00000100:00100000:0.0:1789760695.189898:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495701248 00000100:00100000:0.0:1789760695.189959:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679424972470 time 679463929380 state 0x0 ref 1 00000100:00100000:0.0:1789760695.189975:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679464189796 time 679463929380 state 0x1 ref 1 00000100:00100000:0.0:1789760695.190002:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679464189796 time 679510719717 state 0x0 ref 1 00000100:00100000:0.0:1789760695.190015:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495701248:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760695.190061:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19980288, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:0.0:1789760695.190071:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544d2e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.190086:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=679s v=(0 0) 00010000:00010000:0.0:1789760695.190090:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760695.190096:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544d2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.190109:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544d2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760695.232735:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972195 is committed 00002000:00010000:0.0:1789760695.232970:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19980288 -> 19984384 00010000:00010000:0.0:1789760695.232987:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544d2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.233001:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544d2e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.233016:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544d2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.233024:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760695.233030:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544d2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.233039:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544d2e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.233059:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544d2e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.233067:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544d2e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760695.233154:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495701248:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43135us (44655us total) trans 4294972195 rc 0/0 00000100:00100000:0.0:1789760695.233163:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4907 00000100:00100000:0.0:1789760695.233175:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679464189796 time 679510719717 state 0x2 ref 0 00000100:00100000:0.0:1789760695.237100:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:2.0:1789760695.301300:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972196 is committed 00002000:00010000:1.0:1789760695.301946:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19984384 -> 19988480 00010000:00010000:1.0:1789760695.301961:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544d35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.301974:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544d35 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.301983:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544d35 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.301990:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760695.301996:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544d35 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.302005:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544d35 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.302024:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544d35 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.302032:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544d35 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760695.302129:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495701376:12345-192.168.203.9@tcp:4:dd.0 Request processed in 64798us (65033us total) trans 4294972196 rc 0/0 00000100:00100000:1.0:1789760695.302140:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4908 00000100:00100000:1.0:1789760695.302150:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679510980133 time 679558031211 state 0x2 ref 0 00000100:00100000:2.0:1789760695.308471:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760695.308658:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495701504 00000100:00100000:0.0:1789760695.308668:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495701504 00000100:00100000:0.0:1789760695.308732:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679510980133 time 679558031211 state 0x0 ref 1 00000100:00100000:0.0:1789760695.308751:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679558291627 time 679558031211 state 0x1 ref 1 00000100:00100000:0.0:1789760695.308780:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679558291627 time 679629496396 state 0x0 ref 1 00000100:00100000:0.0:1789760695.308793:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f873480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495701504:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760695.308840:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19988480, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:0.0:1789760695.308849:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544d3c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.308867:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=679s v=(0 0) 00010000:00010000:0.0:1789760695.308872:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760695.308879:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544d3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.308894:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544d3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760695.368988:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972197 is committed 00000100:00100000:2.0:1789760695.377052:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760695.377170:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495701632 00000100:00100000:0.0:1789760695.377179:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495701632 00000100:00100000:0.0:1789760695.377248:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679558291627 time 679629496396 state 0x0 ref 1 00000100:00100000:0.0:1789760695.377263:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679629756812 time 679629496396 state 0x1 ref 1 00000100:00100000:0.0:1789760695.377328:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679629756812 time 679698044817 state 0x0 ref 1 00000100:00100000:0.0:1789760695.377341:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f871500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495701632:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760695.377588:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19992576, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:0.0:1789760695.377599:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d43 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.377616:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=679s v=(0 0) 00010000:00010000:0.0:1789760695.377620:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760695.377627:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.377640:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760695.439984:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972198 is committed 00002000:00010000:0.0:1789760695.440082:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19992576 -> 19996672 00010000:00010000:0.0:1789760695.440097:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.440110:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d43 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.440119:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d43 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.440126:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760695.440131:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d43 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.440140:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d43 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.440161:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d43 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.440169:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354544d43 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760695.440266:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f871500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495701632:12345-192.168.203.9@tcp:4:dd.0 Request processed in 62923us (63218us total) trans 4294972198 rc 0/0 00000100:00100000:0.0:1789760695.440276:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4910 00000100:00100000:0.0:1789760695.440334:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679629756812 time 679698044817 state 0x2 ref 0 00000100:00100000:1.0:1789760695.444196:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00002000:00010000:0.0:1789760695.552875:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19996672 -> 20000768 00010000:00010000:0.0:1789760695.552893:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354544d4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.552907:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354544d4a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.552916:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354544d4a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.552924:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760695.552929:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354544d4a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.552943:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354544d4a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.552962:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354544d4a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.552969:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354544d4a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760695.553067:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03097100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495701760:12345-192.168.203.9@tcp:4:dd.0 Request processed in 108510us (108876us total) trans 4294972199 rc 0/0 00000100:00100000:0.0:1789760695.553076:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4911 00000100:00100000:0.0:1789760695.553086:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679698305233 time 679765260426 state 0x2 ref 0 00000100:00100000:2.0:1789760695.556985:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:2.0:1789760695.630889:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972200 is committed 00002000:00010000:1.0:1789760695.637329:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20000768 -> 20004864 00010000:00010000:1.0:1789760695.637356:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544d51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.637438:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544d51 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.637447:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544d51 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.637453:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760695.637458:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544d51 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.637537:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544d51 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.637561:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544d51 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.637574:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544d51 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760695.637669:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f871f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495701888:12345-192.168.203.9@tcp:4:dd.0 Request processed in 80466us (80688us total) trans 4294972200 rc 0/0 00000100:00100000:1.0:1789760695.637681:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4912 00000100:00100000:1.0:1789760695.637691:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679765520842 time 679877903743 state 0x2 ref 0 00000100:00100000:2.0:1789760695.645148:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760695.645265:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495702016 00000100:00100000:0.0:1789760695.645274:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495702016 00000100:00100000:0.0:1789760695.645354:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679765520842 time 679877903743 state 0x0 ref 1 00000100:00100000:0.0:1789760695.645376:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679878164159 time 679877903743 state 0x1 ref 1 00000100:00100000:0.0:1789760695.645409:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679878164159 time 679966125987 state 0x0 ref 1 00000100:00100000:0.0:1789760695.645421:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495702016:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760695.645467:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20004864, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:0.0:1789760695.645476:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354544d58 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.645493:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=679s v=(0 0) 00010000:00010000:0.0:1789760695.645498:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760695.645504:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354544d58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.645516:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354544d58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760695.714628:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972201 is committed 00002000:00010000:0.0:1789760695.714856:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20004864 -> 20008960 00002000:00010000:0.0:1789760695.714861:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760694 -> 1789760695 00002000:00010000:0.0:1789760695.714864:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760694 -> 1789760695 00010000:00010000:0.0:1789760695.714876:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354544d58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.714889:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354544d58 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.714898:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354544d58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.714905:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760695.714909:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354544d58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.714918:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354544d58 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.714938:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354544d58 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.714946:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354544d58 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760695.715029:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495702016:12345-192.168.203.9@tcp:4:dd.0 Request processed in 69606us (69886us total) trans 4294972201 rc 0/0 00000100:00100000:0.0:1789760695.715039:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4913 00000100:00100000:0.0:1789760695.715048:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679878164159 time 679966125987 state 0x2 ref 0 00000100:00100000:0.0:1789760695.734932:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:0.0:1789760695.805632:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972202 is committed 00002000:00010000:1.0:1789760695.805764:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20008960 -> 20013056 00010000:00010000:1.0:1789760695.805783:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544d5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.805798:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544d5f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.805809:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544d5f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.805816:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760695.805822:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544d5f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.805831:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544d5f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.805854:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544d5f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.805861:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544d5f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760695.805958:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04933480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495702144:12345-192.168.203.9@tcp:4:dd.0 Request processed in 70786us (71031us total) trans 4294972202 rc 0/0 00000100:00100000:1.0:1789760695.805967:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4914 00000100:00100000:1.0:1789760695.805979:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679966386403 time 680055873639 state 0x2 ref 0 00000100:00100000:2.0:1789760695.811001:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760695.811432:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495702272 00000100:00100000:1.0:1789760695.811444:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495702272 00000100:00100000:1.0:1789760695.811501:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 679966386403 time 680055873639 state 0x0 ref 1 00000100:00100000:1.0:1789760695.811518:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680056134055 time 680055873639 state 0x1 ref 1 00000100:00100000:1.0:1789760695.811553:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680056134055 time 680132269741 state 0x0 ref 1 00000100:00100000:1.0:1789760695.811567:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495702272:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760695.811620:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20013056, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:1.0:1789760695.811631:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354544d66 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.811647:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=680s v=(0 0) 00010000:00010000:1.0:1789760695.811652:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760695.811659:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354544d66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.811673:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354544d66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760695.904191:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972203 is committed 00002000:00010000:1.0:1789760695.904347:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20013056 -> 20017152 00010000:00010000:1.0:1789760695.904371:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354544d66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.904392:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354544d66 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.904401:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354544d66 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.904409:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760695.904414:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354544d66 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.904422:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354544d66 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.904440:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354544d66 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.904448:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354544d66 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760695.904527:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495702272:12345-192.168.203.9@tcp:4:dd.0 Request processed in 92959us (93532us total) trans 4294972203 rc 0/0 00000100:00100000:1.0:1789760695.904538:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4915 00000100:00100000:1.0:1789760695.904548:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680056134055 time 680132269741 state 0x2 ref 0 00000100:00100000:0.0:1789760695.916684:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760695.916791:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495702400 00000100:00100000:2.0:1789760695.916800:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495702400 00000100:00100000:2.0:1789760695.916853:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680056134055 time 680132269741 state 0x0 ref 1 00000100:00100000:2.0:1789760695.916867:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680132530157 time 680132269741 state 0x1 ref 1 00000100:00100000:2.0:1789760695.916899:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680132530157 time 680237616363 state 0x0 ref 1 00000100:00100000:2.0:1789760695.916910:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04932680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495702400:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760695.916954:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20017152, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:2.0:1789760695.916963:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354544d6d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.916978:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=680s v=(0 0) 00010000:00010000:2.0:1789760695.916982:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760695.916989:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354544d6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.917002:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354544d6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760695.968547:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972204 is committed 00000100:00100000:0.0:1789760695.978126:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760695.978276:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495702528 00000100:00100000:1.0:1789760695.978318:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495702528 00000100:00100000:1.0:1789760695.978377:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680132530157 time 680237616363 state 0x0 ref 1 00000100:00100000:1.0:1789760695.978401:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680237876779 time 680237616363 state 0x1 ref 1 00000100:00100000:1.0:1789760695.978429:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680237876779 time 680299145657 state 0x0 ref 1 00000100:00100000:1.0:1789760695.978441:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04930a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495702528:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760695.978486:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20021248, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:1.0:1789760695.978495:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354544d74 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.978510:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=680s v=(0 0) 00010000:00010000:1.0:1789760695.978515:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760695.978522:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354544d74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.978535:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354544d74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760696.055097:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972205 is committed 00000100:00100000:2.0:1789760696.059531:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760696.059698:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495702656 00000100:00100000:1.0:1789760696.059718:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495702656 00000100:00100000:1.0:1789760696.059772:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680237876779 time 680299145657 state 0x0 ref 1 00000100:00100000:1.0:1789760696.059787:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680299406073 time 680299145657 state 0x1 ref 1 00000100:00100000:1.0:1789760696.059820:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680299406073 time 680380537221 state 0x0 ref 1 00000100:00100000:1.0:1789760696.059832:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495702656:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760696.059879:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20025344, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:1.0:1789760696.059887:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354544d7b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.059902:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=680s v=(0 0) 00010000:00010000:1.0:1789760696.059907:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760696.059913:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354544d7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.059926:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354544d7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760696.149188:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20025344 -> 20029440 00010000:00010000:2.0:1789760696.149205:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354544d7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.149219:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354544d7b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.149229:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354544d7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.149237:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760696.149242:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354544d7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.149252:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354544d7b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.149271:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354544d7b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.149298:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354544d7b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760696.149416:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495702656:12345-192.168.203.9@tcp:4:dd.0 Request processed in 89582us (89892us total) trans 4294972206 rc 0/0 00000100:00100000:2.0:1789760696.149425:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4918 00000100:00100000:2.0:1789760696.149434:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680299406073 time 680380537221 state 0x2 ref 0 00000100:00100000:1.0:1789760696.153148:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00010000:00010000:1.0:1789760696.163736:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760696.192644:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972207 is committed 00000100:00100000:0.0:1789760696.198860:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00002000:00010000:1.0:1789760696.256101:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20033536 -> 20037632 00010000:00010000:1.0:1789760696.256121:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544d89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.256134:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544d89 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.256144:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544d89 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.256152:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760696.256158:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544d89 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.256167:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544d89 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.256197:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544d89 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.256205:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544d89 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760696.256328:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495702912:12345-192.168.203.9@tcp:4:dd.0 Request processed in 57279us (57475us total) trans 4294972208 rc 0/0 00000100:00100000:1.0:1789760696.256339:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4920 00000100:00100000:1.0:1789760696.256348:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680485037897 time 680519752017 state 0x2 ref 0 00000001:00080000:2.0:1789760696.256604:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972208 is committed 00000100:00100000:2.0:1789760696.263821:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760696.263936:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495703040 00000100:00100000:1.0:1789760696.263947:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495703040 00000100:00100000:1.0:1789760696.264002:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680485037897 time 680519752017 state 0x0 ref 1 00000100:00100000:1.0:1789760696.264017:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680520012433 time 680519752017 state 0x1 ref 1 00000100:00100000:1.0:1789760696.264095:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680520012433 time 680584811132 state 0x0 ref 1 00000100:00100000:1.0:1789760696.264109:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495703040:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760696.264158:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20037632, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:1.0:1789760696.264167:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544d90 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.264194:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=680s v=(0 0) 00010000:00010000:1.0:1789760696.264199:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760696.264205:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544d90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.264220:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544d90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760696.286928:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972209 is committed 00002000:00010000:1.0:1789760696.287022:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20037632 -> 20041728 00010000:00010000:1.0:1789760696.287038:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544d90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.287050:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544d90 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.287059:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544d90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.287067:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760696.287072:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544d90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.287081:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544d90 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.287101:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544d90 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.287109:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544d90 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760696.287202:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495703040:12345-192.168.203.9@tcp:4:dd.0 Request processed in 23090us (23386us total) trans 4294972209 rc 0/0 00000100:00100000:1.0:1789760696.287212:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4921 00000100:00100000:1.0:1789760696.287225:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680520012433 time 680584811132 state 0x2 ref 0 00000100:00100000:0.0:1789760696.290988:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:0.0:1789760696.334088:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972210 is committed 00000100:00100000:0.0:1789760696.338684:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760696.338833:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495703296 00000100:00100000:1.0:1789760696.338842:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495703296 00000100:00100000:1.0:1789760696.338898:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680585071548 time 680612368181 state 0x0 ref 1 00000100:00100000:1.0:1789760696.338912:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680612628597 time 680612368181 state 0x1 ref 1 00000100:00100000:1.0:1789760696.338970:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680612628597 time 680659687161 state 0x0 ref 1 00000100:00100000:1.0:1789760696.338983:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04933b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495703296:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760696.339126:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20045824, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:1.0:1789760696.339135:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544d9e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.339151:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=680s v=(0 0) 00010000:00010000:1.0:1789760696.339156:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760696.339162:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544d9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.339276:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544d9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760696.359494:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20045824 -> 20049920 00010000:00010000:1.0:1789760696.359513:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544d9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.359526:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544d9e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.359536:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544d9e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.359543:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760696.359548:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544d9e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.359558:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544d9e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.359577:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544d9e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.359589:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544d9e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760696.359681:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04933b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495703296:12345-192.168.203.9@tcp:4:dd.0 Request processed in 20688us (20997us total) trans 4294972211 rc 0/0 00000100:00100000:1.0:1789760696.359693:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4923 00000001:00080000:0.0:1789760696.359789:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972211 is committed 00000100:00100000:1.0:1789760696.360167:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680612628597 time 680659687161 state 0x2 ref 0 00000100:00100000:2.0:1789760696.362888:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760696.363074:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495703424 00000100:00100000:1.0:1789760696.363083:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495703424 00000100:00100000:1.0:1789760696.363136:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680612628597 time 680659687161 state 0x0 ref 1 00000100:00100000:1.0:1789760696.363151:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680659947577 time 680659687161 state 0x1 ref 1 00000100:00100000:1.0:1789760696.363184:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680659947577 time 680683901196 state 0x0 ref 1 00000100:00100000:1.0:1789760696.363196:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495703424:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760696.363243:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20049920, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:1.0:1789760696.363252:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354544da5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.363268:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=680s v=(0 0) 00010000:00010000:1.0:1789760696.363272:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760696.363349:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354544da5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.363424:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354544da5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760696.385301:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972212 is committed 00002000:00010000:2.0:1789760696.385396:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20049920 -> 20054016 00010000:00010000:2.0:1789760696.385412:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354544da5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.385425:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354544da5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.385433:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354544da5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.385440:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760696.385445:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354544da5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.385454:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354544da5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.385472:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354544da5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.385480:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354544da5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760696.385568:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495703424:12345-192.168.203.9@tcp:4:dd.0 Request processed in 22369us (22684us total) trans 4294972212 rc 0/0 00000100:00100000:2.0:1789760696.385579:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4924 00000100:00100000:2.0:1789760696.385589:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680659947577 time 680683901196 state 0x2 ref 0 00000100:00100000:2.0:1789760696.389214:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760696.389642:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680684161612 time 680710357152 state 0x0 ref 1 00000100:00100000:0.0:1789760696.389658:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495703552:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760696.389706:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20054016, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:0.0:1789760696.389714:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354544dac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.389730:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=680s v=(0 0) 00010000:00010000:0.0:1789760696.389735:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760696.389751:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354544dac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.389764:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354544dac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760696.435662:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760696.435839:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495703680 00000100:00100000:2.0:1789760696.435849:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495703680 00000100:00100000:2.0:1789760696.435901:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680684161612 time 680710357152 state 0x0 ref 1 00000100:00100000:2.0:1789760696.435916:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680710617568 time 680710357152 state 0x1 ref 1 00000100:00100000:2.0:1789760696.435948:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680710617568 time 680756664316 state 0x0 ref 1 00000100:00100000:2.0:1789760696.435960:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03094700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495703680:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760696.436005:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20058112, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:2.0:1789760696.436013:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544db3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.436034:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=680s v=(0 0) 00010000:00010000:2.0:1789760696.436038:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760696.436044:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544db3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.436058:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544db3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760696.496862:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20058112 -> 20062208 00010000:00010000:2.0:1789760696.496882:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544db3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.497163:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544db3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.497174:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544db3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.497182:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760696.497188:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544db3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.497199:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544db3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.497217:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544db3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.497225:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544db3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760696.497534:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03094700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495703680:12345-192.168.203.9@tcp:4:dd.0 Request processed in 61570us (61874us total) trans 4294972214 rc 0/0 00000100:00100000:2.0:1789760696.497544:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4926 00000100:00100000:2.0:1789760696.497554:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680710617568 time 680756664316 state 0x2 ref 0 00000100:00100000:2.0:1789760696.503140:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760696.503318:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495703808 00000100:00100000:0.0:1789760696.503330:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495703808 00000100:00100000:0.0:1789760696.503402:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680710617568 time 680756664316 state 0x0 ref 1 00000100:00100000:0.0:1789760696.503417:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680756924732 time 680756664316 state 0x1 ref 1 00000100:00100000:1.0:1789760696.503661:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680756924732 time 680824375117 state 0x0 ref 1 00000100:00100000:1.0:1789760696.503677:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903639d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495703808:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760696.503733:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20062208, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:1.0:1789760696.503743:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354544dba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.503758:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=680s v=(0 0) 00010000:00010000:1.0:1789760696.503763:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760696.503768:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354544dba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.503781:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354544dba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760696.587483:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20062208 -> 20066304 00010000:00010000:0.0:1789760696.587499:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354544dba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.587512:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354544dba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.587521:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354544dba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.587534:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760696.587539:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354544dba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.587547:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354544dba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.587568:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354544dba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.587576:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354544dba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.587666:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903639d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495703808:12345-192.168.203.9@tcp:4:dd.0 Request processed in 83987us (84530us total) trans 4294972215 rc 0/0 00000100:00100000:0.0:1789760696.587676:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4927 00000100:00100000:0.0:1789760696.587686:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680756924732 time 680824375117 state 0x2 ref 0 00000100:00100000:2.0:1789760696.592078:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760696.606700:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9290113a6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3318:1876699520297984:0@lo:400:kworker.0 00000100:00100000:2.0:1789760696.606778:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760696.606844:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9290113a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3318:1876699520298496:0@lo:400:kworker.0 00000100:00100000:2.0:1789760696.606871:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760696.606898:0:3318:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1789760696.606904:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9290113a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:59c4e264-9b75-482b-8813-3d1dc124f384:3318:1876699520297856:0@lo:400:kworker.0 00000100:00100000:2.0:1789760696.606932:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760696.606953:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9290113a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3318:1876699520298368:0@lo:400:kworker.0 00000100:00100000:2.0:1789760696.606973:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760696.607003:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520298496 00000100:00100000:2.0:1789760696.607010:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520298496 00000100:00100000:2.0:1789760696.607059:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 206 00000100:00100000:2.0:1789760696.607068:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3318:x1876699520298496:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760696.607117:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3318:x1876699520298496:12345-0@lo:400:kworker.0 Request processed in 50us (245us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760696.607124:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 206 00000100:00100000:2.0:1789760696.607149:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520297856 00000100:00100000:2.0:1789760696.607152:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520297856 00000100:00100000:2.0:1789760696.607172:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 383 00000100:00100000:2.0:1789760696.607178:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290062f5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:59c4e264-9b75-482b-8813-3d1dc124f384+19:3318:x1876699520297856:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760696.607202:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290062f5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:59c4e264-9b75-482b-8813-3d1dc124f384+19:3318:x1876699520297856:12345-0@lo:400:kworker.0 Request processed in 26us (271us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760696.607208:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 383 00000100:00100000:2.0:1789760696.607229:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520297984 00000100:00100000:2.0:1789760696.607232:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520297984 00000100:00100000:2.0:1789760696.607249:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520298368 00000100:00100000:2.0:1789760696.607251:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520298368 00000100:00100000:2.0:1789760696.607268:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 437 00000100:00100000:2.0:1789760696.607274:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3318:x1876699520297984:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760696.607573:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3318:x1876699520297984:12345-0@lo:400:kworker.0 Request processed in 299us (795us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760696.607580:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 437 00000100:00080000:2.0:1789760696.607598:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789760696.607601:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789760696.607626:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9290113a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:59c4e264-9b75-482b-8813-3d1dc124f384:3318:1876699520297856:0@lo:400:kworker.0 00000100:00100000:2.0:1789760696.607639:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9290113a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3318:1876699520298368:0@lo:400:kworker.0 00000100:00100000:2.0:1789760696.607648:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9290113a6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3318:1876699520297984:0@lo:400:kworker.0 00000100:00100000:2.0:1789760696.607655:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9290113a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3318:1876699520298496:0@lo:400:kworker.0 00000100:00100000:0.0:1789760696.610354:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9290113a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520298112:0@lo:400:kworker.0 00000100:00100000:0.0:1789760696.610433:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760696.610483:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789760696.610490:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9290113a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520298240:0@lo:400:kworker.0 00000100:00100000:0.0:1789760696.610516:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760696.610622:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520298112 00000100:00100000:0.0:1789760696.610630:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520298112 00000100:00100000:0.0:1789760696.610683:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 439 00000100:00100000:0.0:1789760696.610690:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04930000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520298112:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760696.610750:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04930000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520298112:12345-0@lo:400:kworker.0 Request processed in 49us (306us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760696.610757:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 439 00000100:00100000:0.0:1789760696.610799:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9290113a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520298112:0@lo:400:kworker.0 00000100:00080000:1.0:1789760696.610876:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789760696.613776:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789760696.648701:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20066304 -> 20070400 00010000:00010000:2.0:1789760696.648716:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544dc1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.648728:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544dc1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.648738:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544dc1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.648745:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760696.648749:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544dc1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.648757:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544dc1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.648776:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544dc1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.648784:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544dc1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760696.648874:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495703936:12345-192.168.203.9@tcp:4:dd.0 Request processed in 56590us (56799us total) trans 4294972216 rc 0/0 00000100:00100000:2.0:1789760696.648884:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4928 00000100:00100000:2.0:1789760696.648904:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680824635533 time 680912986354 state 0x2 ref 0 00000100:00100000:1.0:1789760696.654596:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760696.655866:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495704064 00000100:00100000:2.0:1789760696.655877:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495704064 00000100:00100000:2.0:1789760696.655937:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680824635533 time 680912986354 state 0x0 ref 1 00000100:00100000:2.0:1789760696.655952:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680913246770 time 680912986354 state 0x1 ref 1 00000100:00100000:2.0:1789760696.655987:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680913246770 time 680976703433 state 0x0 ref 1 00000100:00100000:2.0:1789760696.655999:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495704064:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760696.656043:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20070400, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:2.0:1789760696.656052:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544dc8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.656068:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=680s v=(0 0) 00010000:00010000:2.0:1789760696.656072:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760696.656078:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544dc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.656092:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544dc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760696.704174:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20070400 -> 20074496 00002000:00010000:0.0:1789760696.704182:0:6714:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760695 -> 1789760696 00002000:00010000:0.0:1789760696.704185:0:6714:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760695 -> 1789760696 00010000:00010000:0.0:1789760696.704200:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544dc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.704211:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544dc8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.704219:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544dc8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.704227:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760696.704232:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544dc8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.704240:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544dc8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.704261:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544dc8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.704268:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544dc8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.704564:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495704064:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48561us (49971us total) trans 4294972217 rc 0/0 00000100:00100000:0.0:1789760696.704575:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4929 00000100:00100000:0.0:1789760696.704585:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680913246770 time 680976703433 state 0x2 ref 0 00000001:00080000:1.0:1789760696.708121:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972217 is committed 00000100:00100000:0.0:1789760696.708268:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495704192 00000100:00100000:0.0:1789760696.708297:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495704192 00000100:00100000:0.0:1789760696.708364:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680913246770 time 680976703433 state 0x0 ref 1 00000100:00100000:0.0:1789760696.708387:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680976963849 time 680976703433 state 0x1 ref 1 00000100:00100000:0.0:1789760696.708409:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680976963849 time 681029126017 state 0x0 ref 1 00000100:00100000:0.0:1789760696.708420:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495704192:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760696.708468:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20074496, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:0.0:1789760696.708477:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354544dcf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.708492:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=681s v=(0 0) 00010000:00010000:0.0:1789760696.708496:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760696.708501:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354544dcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.708521:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354544dcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760696.757088:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20074496 -> 20078592 00010000:00010000:1.0:1789760696.757106:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354544dcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.757119:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354544dcf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.757128:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354544dcf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.757135:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760696.757140:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354544dcf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.757149:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354544dcf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.757168:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354544dcf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.757176:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354544dcf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760696.757273:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495704192:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48850us (49318us total) trans 4294972218 rc 0/0 00000100:00100000:1.0:1789760696.757318:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4930 00000100:00100000:1.0:1789760696.757327:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680976963849 time 681029126017 state 0x2 ref 0 00000001:00080000:1.0:1789760696.759346:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972218 is committed 00000100:00100000:2.0:1789760696.761168:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495704320 00000100:00100000:2.0:1789760696.761177:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495704320 00000100:00100000:2.0:1789760696.761232:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 680976963849 time 681029126017 state 0x0 ref 1 00000100:00100000:2.0:1789760696.761247:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681029386433 time 681029126017 state 0x1 ref 1 00000100:00100000:2.0:1789760696.761310:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681029386433 time 681082026991 state 0x0 ref 1 00000100:00100000:2.0:1789760696.761323:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495704320:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760696.761388:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20078592, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:2.0:1789760696.761400:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544dd6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.761415:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=681s v=(0 0) 00010000:00010000:2.0:1789760696.761419:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760696.761425:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544dd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.761438:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544dd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760696.793142:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20078592 -> 20082688 00010000:00010000:2.0:1789760696.793157:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544dd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.793171:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544dd6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.793180:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544dd6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.793187:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760696.793193:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544dd6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.793202:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544dd6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.793220:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544dd6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.793228:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544dd6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.793572:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495704320:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32247us (32495us total) trans 4294972219 rc 0/0 00000100:00100000:0.0:1789760696.793583:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4931 00000100:00100000:0.0:1789760696.793592:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681029386433 time 681082026991 state 0x2 ref 0 00000100:00100000:2.0:1789760696.797436:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760696.797496:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495704448 00000100:00100000:1.0:1789760696.797505:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495704448 00000100:00100000:1.0:1789760696.797549:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681029386433 time 681082026991 state 0x0 ref 1 00000100:00100000:1.0:1789760696.797563:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681082287407 time 681082026991 state 0x1 ref 1 00000100:00100000:1.0:1789760696.797589:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681082287407 time 681118306619 state 0x0 ref 1 00000100:00100000:1.0:1789760696.797599:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495704448:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760696.797640:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20082688, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:1.0:1789760696.797648:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354544ddd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.797664:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=681s v=(0 0) 00010000:00010000:1.0:1789760696.797668:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760696.797674:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354544ddd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.797687:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354544ddd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760696.853785:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972220 is committed 00000100:00100000:2.0:1789760696.857693:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495704576 00000100:00100000:2.0:1789760696.857711:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495704576 00000100:00100000:2.0:1789760696.857766:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681082287407 time 681118306619 state 0x0 ref 1 00000100:00100000:2.0:1789760696.857781:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681118567035 time 681118306619 state 0x1 ref 1 00000100:00100000:2.0:1789760696.857813:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681118567035 time 681178529669 state 0x0 ref 1 00000100:00100000:2.0:1789760696.857825:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495704576:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760696.857867:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20086784, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:2.0:1789760696.857875:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544de4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.857895:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=681s v=(0 0) 00010000:00010000:2.0:1789760696.857899:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760696.857905:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544de4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.857918:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544de4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760696.907308:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20086784 -> 20090880 00010000:00010000:1.0:1789760696.907336:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544de4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.907349:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544de4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.907365:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544de4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.907380:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760696.907386:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544de4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.907395:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544de4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.907412:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544de4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.907419:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544de4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760696.907519:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495704576:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49690us (50008us total) trans 4294972221 rc 0/0 00000100:00100000:1.0:1789760696.907533:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4933 00000100:00100000:1.0:1789760696.907544:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681118567035 time 681178529669 state 0x2 ref 0 00000100:00100000:2.0:1789760696.911559:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760696.911651:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495704704 00000100:00100000:0.0:1789760696.911660:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495704704 00000100:00100000:0.0:1789760696.911877:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681118567035 time 681178529669 state 0x0 ref 1 00000100:00100000:0.0:1789760696.911892:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681178790085 time 681178529669 state 0x1 ref 1 00000100:00100000:0.0:1789760696.911923:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681178790085 time 681232640297 state 0x0 ref 1 00000100:00100000:0.0:1789760696.911936:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495704704:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760696.911983:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20090880, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:0.0:1789760696.911992:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354544deb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.912007:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=681s v=(0 0) 00010000:00010000:0.0:1789760696.912012:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760696.912019:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354544deb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.912039:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354544deb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.960201:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760696.960684:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681178790085 time 681232640297 state 0x0 ref 1 00000100:00100000:2.0:1789760696.960764:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681232900713 time 681232640297 state 0x1 ref 1 00000100:00100000:2.0:1789760696.960826:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681232900713 time 681281542347 state 0x0 ref 1 00000100:00100000:2.0:1789760696.960839:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04931f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495704832:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760696.960884:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20094976, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:2.0:1789760696.960894:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544df2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.960910:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=681s v=(0 0) 00010000:00010000:2.0:1789760696.960915:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760696.960921:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544df2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.960940:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544df2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760697.003835:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972223 is committed 00000100:00100000:1.0:1789760697.010047:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495704960 00000100:00100000:1.0:1789760697.010057:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495704960 00000100:00100000:1.0:1789760697.010167:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681232900713 time 681281542347 state 0x0 ref 1 00000100:00100000:1.0:1789760697.010182:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681281802763 time 681281542347 state 0x1 ref 1 00000100:00100000:1.0:1789760697.010212:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681281802763 time 681330928995 state 0x0 ref 1 00000100:00100000:1.0:1789760697.010225:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495704960:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760697.010268:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20099072, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:1.0:1789760697.010277:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354544df9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.010309:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=681s v=(0 0) 00010000:00010000:1.0:1789760697.010325:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760697.010331:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354544df9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.010344:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354544df9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760697.014252:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760697.077472:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972224 is committed 00000100:00100000:0.0:1789760697.082321:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:0.0:1789760697.140300:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972225 is committed 00002000:00010000:1.0:1789760697.140680:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20103168 -> 20107264 00010000:00010000:1.0:1789760697.140763:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544e00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.140778:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544e00 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.140787:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544e00 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.140794:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760697.140799:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544e00 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.140808:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544e00 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.140827:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544e00 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.140834:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544e00 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760697.141108:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04933480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495705088:12345-192.168.203.9@tcp:4:dd.0 Request processed in 58425us (58832us total) trans 4294972225 rc 0/0 00000100:00100000:1.0:1789760697.141119:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4937 00000100:00100000:1.0:1789760697.141129:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681331189411 time 681403382350 state 0x2 ref 0 00000100:00100000:0.0:1789760697.150711:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495705216 00000100:00100000:0.0:1789760697.150722:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495705216 00000100:00100000:0.0:1789760697.150774:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681331189411 time 681403382350 state 0x0 ref 1 00000100:00100000:0.0:1789760697.150789:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681403642766 time 681403382350 state 0x1 ref 1 00000100:00100000:0.0:1789760697.150819:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681403642766 time 681471535504 state 0x0 ref 1 00000100:00100000:0.0:1789760697.150831:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495705216:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760697.150875:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20107264, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:0.0:1789760697.150883:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544e07 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.150910:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=681s v=(0 0) 00010000:00010000:0.0:1789760697.150915:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760697.150922:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544e07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.150940:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544e07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.183387:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760697.222846:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20107264 -> 20111360 00010000:00010000:0.0:1789760697.222867:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544e07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.222879:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544e07 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.222899:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544e07 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.222907:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760697.222911:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544e07 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.222919:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544e07 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.222939:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544e07 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.222946:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544e07 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760697.223037:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495705216:12345-192.168.203.9@tcp:4:dd.0 Request processed in 72203us (72545us total) trans 4294972226 rc 0/0 00000100:00100000:0.0:1789760697.223047:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4938 00000100:00100000:0.0:1789760697.223058:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681403642766 time 681471535504 state 0x2 ref 0 00000001:00080000:1.0:1789760697.223194:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972226 is committed 00000100:00100000:1.0:1789760697.226598:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495705344 00000100:00100000:1.0:1789760697.226607:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495705344 00000100:00100000:1.0:1789760697.226661:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681403642766 time 681471535504 state 0x0 ref 1 00000100:00100000:1.0:1789760697.226677:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681471795920 time 681471535504 state 0x1 ref 1 00000100:00100000:1.0:1789760697.226714:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681471795920 time 681547430260 state 0x0 ref 1 00000100:00100000:1.0:1789760697.226726:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495705344:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760697.226778:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20111360, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:1.0:1789760697.226788:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354544e0e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.226803:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=681s v=(0 0) 00010000:00010000:1.0:1789760697.226808:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760697.226814:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354544e0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.226827:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354544e0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760697.266308:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972227 is committed 00000100:00100000:1.0:1789760697.270945:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495705472 00000100:00100000:1.0:1789760697.270954:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495705472 00000100:00100000:1.0:1789760697.271007:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681471795920 time 681547430260 state 0x0 ref 1 00000100:00100000:1.0:1789760697.271023:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681547690676 time 681547430260 state 0x1 ref 1 00000100:00100000:1.0:1789760697.271054:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681547690676 time 681591771288 state 0x0 ref 1 00000100:00100000:1.0:1789760697.271065:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495705472:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760697.271110:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20115456, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:1.0:1789760697.271119:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354544e15 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.271135:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=681s v=(0 0) 00010000:00010000:1.0:1789760697.271140:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760697.271146:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354544e15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.271160:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354544e15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760697.307991:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972228 is committed 00002000:00010000:0.0:1789760697.309562:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20115456 -> 20119552 00010000:00010000:0.0:1789760697.309582:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354544e15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.309594:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354544e15 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.309604:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354544e15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.309612:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760697.309617:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354544e15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.309626:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354544e15 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.309645:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354544e15 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.309663:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354544e15 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760697.309754:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495705472:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38685us (39030us total) trans 4294972228 rc 0/0 00000100:00100000:0.0:1789760697.309765:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4940 00000100:00100000:0.0:1789760697.309774:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681547690676 time 681591771288 state 0x2 ref 0 00000100:00100000:2.0:1789760697.316970:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760697.317051:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495705600 00000100:00100000:0.0:1789760697.317061:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495705600 00000100:00100000:0.0:1789760697.317115:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681547690676 time 681591771288 state 0x0 ref 1 00000100:00100000:0.0:1789760697.317129:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681592031704 time 681591771288 state 0x1 ref 1 00000100:00100000:0.0:1789760697.317158:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681592031704 time 681637875187 state 0x0 ref 1 00000100:00100000:0.0:1789760697.317170:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495705600:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760697.317218:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20119552, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:0.0:1789760697.317228:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544e1c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.317244:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=681s v=(0 0) 00010000:00010000:0.0:1789760697.317249:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760697.317255:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544e1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.317274:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544e1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760697.345341:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20119552 -> 20123648 00010000:00010000:0.0:1789760697.345369:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544e1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.345393:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544e1c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.345402:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544e1c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.345409:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760697.345413:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544e1c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.345422:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544e1c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.345440:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544e1c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.345447:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544e1c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760697.345538:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495705600:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28364us (28573us total) trans 4294972229 rc 0/0 00000100:00100000:0.0:1789760697.345549:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4941 00000100:00100000:0.0:1789760697.345560:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681592031704 time 681637875187 state 0x2 ref 0 00000001:00080000:2.0:1789760697.346800:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972229 is committed 00000100:00100000:2.0:1789760697.350605:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:2.0:1789760697.401337:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972230 is committed 00000100:00100000:0.0:1789760697.402152:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495705856 00000100:00100000:0.0:1789760697.402160:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495705856 00000100:00100000:0.0:1789760697.402204:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681638135603 time 681671530350 state 0x0 ref 1 00000100:00100000:0.0:1789760697.402215:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681671790766 time 681671530350 state 0x1 ref 1 00000100:00100000:0.0:1789760697.402237:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681671790766 time 681722955396 state 0x0 ref 1 00000100:00100000:0.0:1789760697.402245:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495705856:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760697.402308:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20127744, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:0.0:1789760697.402317:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544e2a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.402332:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=681s v=(0 0) 00010000:00010000:0.0:1789760697.402336:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760697.402342:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544e2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.402365:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544e2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760697.439046:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972231 is committed 00002000:00010000:0.0:1789760697.439156:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20127744 -> 20131840 00010000:00010000:0.0:1789760697.439170:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544e2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.439181:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544e2a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.439189:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544e2a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.439195:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760697.439200:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544e2a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.439208:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544e2a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.439225:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544e2a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.439232:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354544e2a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760697.439517:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495705856:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37123us (37375us total) trans 4294972231 rc 0/0 00000100:00100000:0.0:1789760697.439530:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4943 00000100:00100000:0.0:1789760697.439541:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681671790766 time 681722955396 state 0x2 ref 0 00000100:00100000:0.0:1789760697.443311:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:0.0:1789760697.477157:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972232 is committed 00000100:00100000:0.0:1789760697.482062:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:0.0:1789760697.514005:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972233 is committed 00000100:00100000:0.0:1789760697.517569:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00002000:00010000:0.0:1789760697.543912:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20140032 -> 20144128 00010000:00010000:0.0:1789760697.543932:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544e3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.543944:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544e3f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.543952:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544e3f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.543958:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760697.543963:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544e3f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.543972:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544e3f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.543992:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544e3f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.544000:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544e3f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760697.544085:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495706240:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26176us (26519us total) trans 4294972234 rc 0/0 00000100:00100000:0.0:1789760697.544095:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4946 00000100:00100000:0.0:1789760697.544104:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681803287231 time 681838609677 state 0x2 ref 0 00000001:00080000:0.0:1789760697.544143:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972234 is committed 00000100:00100000:2.0:1789760697.548017:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:0.0:1789760697.575170:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972235 is committed 00000100:00100000:0.0:1789760697.579026:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:1.0:1789760697.610062:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972236 is committed 00000100:00100000:1.0:1789760697.614163:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00002000:00010000:0.0:1789760697.660900:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20152320 -> 20156416 00010000:00010000:0.0:1789760697.660919:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544e54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.660932:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544e54 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.660942:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544e54 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.660950:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760697.660959:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544e54 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760697.660967:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972237 is committed 00010000:00010000:0.0:1789760697.660972:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544e54 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.660990:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544e54 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.660999:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544e54 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760697.661089:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495706624:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46435us (46930us total) trans 4294972237 rc 0/0 00000100:00100000:0.0:1789760697.661099:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4949 00000100:00100000:0.0:1789760697.661109:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681900226983 time 681935355200 state 0x2 ref 0 00000100:00100000:2.0:1789760697.664914:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:0.0:1789760697.709403:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972238 is committed 00000100:00100000:1.0:1789760697.717083:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495706880 00000100:00100000:1.0:1789760697.717094:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495706880 00000100:00100000:1.0:1789760697.717151:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681935615616 time 681985838483 state 0x0 ref 1 00000100:00100000:1.0:1789760697.717168:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681986098899 time 681985838483 state 0x1 ref 1 00000100:00100000:1.0:1789760697.717203:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681986098899 time 682037920077 state 0x0 ref 1 00000100:00100000:1.0:1789760697.717216:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495706880:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760697.717265:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20160512, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:1.0:1789760697.717274:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354544e62 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.717327:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=682s v=(0 0) 00010000:00010000:1.0:1789760697.717332:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760697.717338:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354544e62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.717352:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354544e62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760697.765057:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972239 is committed 00000100:00100000:1.0:1789760697.772585:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760697.772660:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495707008 00000100:00100000:0.0:1789760697.772670:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495707008 00000100:00100000:0.0:1789760697.772722:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 681986098899 time 682037920077 state 0x0 ref 1 00000100:00100000:0.0:1789760697.772736:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682038180493 time 682037920077 state 0x1 ref 1 00000100:00100000:0.0:1789760697.772778:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682038180493 time 682093494376 state 0x0 ref 1 00000100:00100000:0.0:1789760697.772791:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03097b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495707008:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760697.772838:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20164608, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:0.0:1789760697.772847:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354544e69 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.772861:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=682s v=(0 0) 00010000:00010000:0.0:1789760697.772865:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760697.772872:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354544e69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.772886:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354544e69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760697.816004:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972240 is committed 00000100:00100000:1.0:1789760697.819153:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495707136 00000100:00100000:1.0:1789760697.819162:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495707136 00000100:00100000:1.0:1789760697.819216:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682038180493 time 682093494376 state 0x0 ref 1 00000100:00100000:1.0:1789760697.819230:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682093754792 time 682093494376 state 0x1 ref 1 00000100:00100000:1.0:1789760697.819266:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682093754792 time 682139982557 state 0x0 ref 1 00000100:00100000:1.0:1789760697.819307:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495707136:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760697.819363:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20168704, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:1.0:1789760697.819382:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354544e70 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.819398:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=682s v=(0 0) 00010000:00010000:1.0:1789760697.819402:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760697.819408:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354544e70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.819420:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354544e70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760697.843434:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760697.843607:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495707392 00000100:00100000:2.0:1789760697.843618:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495707392 00000100:00100000:2.0:1789760697.843686:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 440 00000100:00100000:2.0:1789760697.843706:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03097100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:17022965-1e4f-46fb-915c-61730096b8e2+8:2357:x1876699495707392:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:2.0:1789760697.843771:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03097100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:17022965-1e4f-46fb-915c-61730096b8e2+8:2357:x1876699495707392:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 67us (347us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760697.843778:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 440 00000100:00100000:1.0:1789760697.844672:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760697.844728:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760697.844760:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495707520 00000100:00100000:1.0:1789760697.844766:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495707520 00000100:00100000:1.0:1789760697.844805:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 441 00000100:00100000:1.0:1789760697.844814:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03094380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2357:x1876699495707520:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:1.0:1789760697.844849:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03094380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2357:x1876699495707520:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 36us (121us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760697.844855:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 441 00000001:00080000:2.0:1789760697.874125:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972241 is committed 00000001:00080000:2.0:1789760697.919146:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972242 is committed 00000100:00100000:0.0:1789760697.924084:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:0.0:1789760697.973145:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972243 is committed 00002000:00010000:1.0:1789760697.973262:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20176896 -> 20180992 00010000:00010000:1.0:1789760697.973340:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354544e7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.973413:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354544e7e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.973425:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354544e7e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.973546:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760697.973552:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354544e7e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.973561:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354544e7e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.973579:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354544e7e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.973587:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354544e7e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760697.973759:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04933b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495707776:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49371us (49679us total) trans 4294972243 rc 0/0 00000100:00100000:1.0:1789760697.973770:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4955 00000100:00100000:1.0:1789760697.973780:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682195715349 time 682245080595 state 0x2 ref 0 00000100:00100000:0.0:1789760697.980153:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495707904 00000100:00100000:0.0:1789760697.980162:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495707904 00000100:00100000:0.0:1789760697.980212:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682195715349 time 682245080595 state 0x0 ref 1 00000100:00100000:0.0:1789760697.980227:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682245341011 time 682245080595 state 0x1 ref 1 00000100:00100000:0.0:1789760697.980261:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682245341011 time 682300978624 state 0x0 ref 1 00000100:00100000:0.0:1789760697.980273:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495707904:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760697.980335:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20180992, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:0.0:1789760697.980355:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354544e85 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.980379:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=682s v=(0 0) 00010000:00010000:0.0:1789760697.980385:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760697.980393:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354544e85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.980407:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354544e85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760698.039775:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972244 is committed 00000100:00100000:1.0:1789760698.040197:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00002000:00010000:1.0:1789760698.087949:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20185088 -> 20189184 00010000:00010000:1.0:1789760698.087969:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544e8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.087987:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544e8c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.087996:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544e8c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.088004:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760698.088010:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544e8c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.088027:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544e8c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.088046:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544e8c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.088054:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544e8c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760698.088240:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03094700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495708032:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46258us (48043us total) trans 4294972245 rc 0/0 00000100:00100000:1.0:1789760698.088251:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4957 00000100:00100000:1.0:1789760698.088261:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682301239040 time 682362681639 state 0x2 ref 0 00000001:00080000:2.0:1789760698.091041:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972245 is committed 00000100:00100000:0.0:1789760698.092825:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760698.092934:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495708160 00000100:00100000:1.0:1789760698.092944:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495708160 00000100:00100000:1.0:1789760698.092993:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682301239040 time 682362681639 state 0x0 ref 1 00000100:00100000:1.0:1789760698.093008:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682362942055 time 682362681639 state 0x1 ref 1 00000100:00100000:1.0:1789760698.093046:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682362942055 time 682413763579 state 0x0 ref 1 00000100:00100000:1.0:1789760698.093059:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04933100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495708160:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760698.093104:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20189184, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:1.0:1789760698.093112:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354544e93 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.093230:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=682s v=(0 0) 00010000:00010000:1.0:1789760698.093236:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760698.093241:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354544e93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.093254:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354544e93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760698.136384:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20189184 -> 20193280 00010000:00010000:1.0:1789760698.136404:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354544e93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.136417:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354544e93 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.136431:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354544e93 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.136439:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760698.136444:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354544e93 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.136453:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354544e93 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.136484:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354544e93 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.136494:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354544e93 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760698.136585:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04933100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495708160:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43524us (43763us total) trans 4294972246 rc 0/0 00000100:00100000:1.0:1789760698.136596:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4958 00000100:00100000:1.0:1789760698.136606:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682362942055 time 682413763579 state 0x2 ref 0 00000001:00080000:0.0:1789760698.140092:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972246 is committed 00000100:00100000:0.0:1789760698.141570:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00010000:00010000:2.0:1789760698.208082:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760698.208763:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972247 is committed 00000100:00100000:2.0:1789760698.214924:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760698.215487:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495708416 00000100:00100000:1.0:1789760698.215498:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495708416 00000100:00100000:1.0:1789760698.215552:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682414023995 time 682462450901 state 0x0 ref 1 00000100:00100000:1.0:1789760698.215569:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682462711317 time 682462450901 state 0x1 ref 1 00000100:00100000:1.0:1789760698.215609:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682462711317 time 682536325619 state 0x0 ref 1 00000100:00100000:1.0:1789760698.215622:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495708416:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760698.215672:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20197376, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:1.0:1789760698.215683:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354544ea1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.215711:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=682s v=(0 0) 00010000:00010000:1.0:1789760698.215716:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760698.215722:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354544ea1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.215736:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354544ea1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760698.260065:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20197376 -> 20201472 00010000:00010000:1.0:1789760698.260080:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354544ea1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.260093:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354544ea1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.260103:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354544ea1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.260116:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760698.260121:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354544ea1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.260130:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354544ea1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.260150:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354544ea1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.260158:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354544ea1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760698.260249:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495708416:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44624us (45328us total) trans 4294972248 rc 0/0 00000100:00100000:2.0:1789760698.263808:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760698.264738:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4960 00000100:00100000:1.0:1789760698.264754:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682462711317 time 682536325619 state 0x2 ref 0 00000100:00100000:1.0:1789760698.264795:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495708544 00000100:00100000:1.0:1789760698.264803:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495708544 00000100:00100000:1.0:1789760698.265051:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682462711317 time 682536325619 state 0x0 ref 1 00000100:00100000:1.0:1789760698.265066:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682536586035 time 682536325619 state 0x1 ref 1 00000100:00100000:1.0:1789760698.265090:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682536586035 time 682585807927 state 0x0 ref 1 00000100:00100000:1.0:1789760698.265105:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495708544:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760698.265155:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20201472, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:1.0:1789760698.265164:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354544ea8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.265179:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=682s v=(0 0) 00010000:00010000:1.0:1789760698.265184:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760698.265191:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354544ea8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.265205:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354544ea8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760698.290816:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20201472 -> 20205568 00010000:00010000:1.0:1789760698.290834:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354544ea8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.290904:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354544ea8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.290914:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354544ea8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.290921:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760698.290926:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354544ea8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.290935:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354544ea8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.290955:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354544ea8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.290962:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354544ea8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760698.291044:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495708544:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25936us (27241us total) trans 4294972249 rc 0/0 00000100:00100000:1.0:1789760698.291054:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4961 00000100:00100000:1.0:1789760698.291063:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682536586035 time 682585807927 state 0x2 ref 0 00000001:00080000:2.0:1789760698.292767:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972249 is committed 00000100:00100000:0.0:1789760698.297422:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495708672 00000100:00100000:0.0:1789760698.297692:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495708672 00000100:00100000:2.0:1789760698.297893:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682536586035 time 682585807927 state 0x0 ref 1 00000100:00100000:2.0:1789760698.297910:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682586068343 time 682585807927 state 0x1 ref 1 00000100:00100000:2.0:1789760698.297977:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682586068343 time 682618692257 state 0x0 ref 1 00000100:00100000:2.0:1789760698.297993:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495708672:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760698.298060:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20205568, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:2.0:1789760698.298070:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544eaf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.298090:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=682s v=(0 0) 00010000:00010000:2.0:1789760698.298095:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760698.298102:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544eaf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.298116:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544eaf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760698.339919:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20205568 -> 20209664 00010000:00010000:2.0:1789760698.339938:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544eaf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.339951:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544eaf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.339966:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544eaf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.339974:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760698.339979:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544eaf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.339988:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544eaf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.340007:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544eaf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.340015:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544eaf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760698.340100:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495708672:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42107us (42973us total) trans 4294972250 rc 0/0 00000100:00100000:2.0:1789760698.340111:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4962 00000100:00100000:2.0:1789760698.340121:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682586068343 time 682618692257 state 0x2 ref 0 00000001:00080000:2.0:1789760698.340166:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972250 is committed 00000100:00100000:2.0:1789760698.344829:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00002000:00010000:2.0:1789760698.410129:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20209664 -> 20213760 00010000:00010000:2.0:1789760698.410150:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544eb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.410164:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544eb6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.410174:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544eb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.410182:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760698.410187:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544eb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.410197:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544eb6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.410220:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544eb6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.410229:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544eb6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760698.410356:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495708800:12345-192.168.203.9@tcp:4:dd.0 Request processed in 65326us (65532us total) trans 4294972251 rc 0/0 00000100:00100000:2.0:1789760698.410375:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4963 00000100:00100000:2.0:1789760698.410385:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682618952673 time 682665731786 state 0x2 ref 0 00000001:00080000:2.0:1789760698.410431:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972251 is committed 00000100:00100000:1.0:1789760698.414272:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:0.0:1789760698.442377:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972252 is committed 00002000:00010000:2.0:1789760698.442486:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20213760 -> 20217856 00010000:00010000:2.0:1789760698.442502:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544ebd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.442516:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544ebd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.442525:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544ebd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.442533:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760698.442539:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544ebd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.442547:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544ebd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.442567:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544ebd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.442575:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354544ebd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760698.442666:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495708928:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28155us (28397us total) trans 4294972252 rc 0/0 00000100:00100000:2.0:1789760698.442676:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4964 00000100:00100000:2.0:1789760698.442696:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682665992202 time 682735212082 state 0x2 ref 0 00000100:00100000:0.0:1789760698.447206:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760698.447421:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495709056 00000100:00100000:2.0:1789760698.447431:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495709056 00000100:00100000:2.0:1789760698.447483:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682665992202 time 682735212082 state 0x0 ref 1 00000100:00100000:2.0:1789760698.447498:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682735472498 time 682735212082 state 0x1 ref 1 00000100:00100000:2.0:1789760698.447569:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682735472498 time 682768285184 state 0x0 ref 1 00000100:00100000:2.0:1789760698.447582:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495709056:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760698.447634:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20217856, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:2.0:1789760698.447644:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544ec4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.447661:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=682s v=(0 0) 00010000:00010000:2.0:1789760698.447665:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760698.447672:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544ec4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.447699:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544ec4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760698.490862:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20217856 -> 20221952 00010000:00010000:2.0:1789760698.490876:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544ec4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.490890:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544ec4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.490899:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544ec4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.490906:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760698.490911:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544ec4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.490919:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544ec4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.490938:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544ec4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.491155:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544ec4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760698.491326:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495709056:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43699us (44083us total) trans 4294972253 rc 0/0 00000100:00100000:2.0:1789760698.491336:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4965 00000100:00100000:2.0:1789760698.491356:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682735472498 time 682768285184 state 0x2 ref 0 00000100:00100000:0.0:1789760698.497184:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760698.497264:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495709184 00000100:00100000:2.0:1789760698.497274:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495709184 00000100:00100000:2.0:1789760698.497539:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682735472498 time 682768285184 state 0x0 ref 1 00000100:00100000:2.0:1789760698.497556:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682768545600 time 682768285184 state 0x1 ref 1 00000100:00100000:2.0:1789760698.497591:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682768545600 time 682818307709 state 0x0 ref 1 00000100:00100000:2.0:1789760698.497605:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495709184:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760698.497651:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20221952, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:2.0:1789760698.497661:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ecb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.497676:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=682s v=(0 0) 00010000:00010000:2.0:1789760698.497681:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760698.497813:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ecb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.497829:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ecb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760698.544207:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20221952 -> 20226048 00010000:00010000:2.0:1789760698.544227:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ecb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.544239:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ecb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.544247:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ecb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.544254:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760698.544260:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ecb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.544269:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ecb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.544314:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ecb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.544323:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ecb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760698.544417:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495709184:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46810us (47237us total) trans 4294972254 rc 0/0 00000100:00100000:2.0:1789760698.544428:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4966 00000100:00100000:2.0:1789760698.544439:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682768545600 time 682818307709 state 0x2 ref 0 00000100:00100000:0.0:1789760698.553383:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760698.555638:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495709312 00000100:00100000:2.0:1789760698.555653:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495709312 00000100:00100000:2.0:1789760698.555717:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682768545600 time 682818307709 state 0x0 ref 1 00000100:00100000:2.0:1789760698.555732:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682818568125 time 682818307709 state 0x1 ref 1 00000100:00100000:2.0:1789760698.555763:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682818568125 time 682876479519 state 0x0 ref 1 00000100:00100000:2.0:1789760698.555775:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495709312:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760698.555824:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20226048, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:2.0:1789760698.555834:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ed2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.555850:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=682s v=(0 0) 00010000:00010000:2.0:1789760698.555855:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760698.555861:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ed2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.555875:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ed2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760698.638351:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972255 is committed 00002000:00010000:2.0:1789760698.638549:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20226048 -> 20230144 00010000:00010000:2.0:1789760698.638565:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ed2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.638578:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ed2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.638587:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ed2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.638595:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760698.638600:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ed2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.638609:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ed2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.638634:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ed2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.638642:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ed2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760698.638832:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495709312:12345-192.168.203.9@tcp:4:dd.0 Request processed in 83054us (85464us total) trans 4294972255 rc 0/0 00000100:00100000:2.0:1789760698.638843:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4967 00000100:00100000:2.0:1789760698.638853:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682818568125 time 682876479519 state 0x2 ref 0 00000100:00100000:0.0:1789760698.646786:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760698.646880:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495709440 00000100:00100000:2.0:1789760698.646890:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495709440 00000100:00100000:2.0:1789760698.646944:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682818568125 time 682876479519 state 0x0 ref 1 00000100:00100000:2.0:1789760698.646960:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682876739935 time 682876479519 state 0x1 ref 1 00000100:00100000:2.0:1789760698.646993:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682876739935 time 682967709569 state 0x0 ref 1 00000100:00100000:2.0:1789760698.647004:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495709440:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760698.647063:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20230144, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:2.0:1789760698.647072:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ed9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.647089:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=682s v=(0 0) 00010000:00010000:2.0:1789760698.647094:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760698.647101:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ed9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.647117:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ed9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760698.685269:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20230144 -> 20234240 00002000:00010000:2.0:1789760698.685305:0:6714:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760697 -> 1789760698 00002000:00010000:2.0:1789760698.685308:0:6714:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760697 -> 1789760698 00010000:00010000:2.0:1789760698.685321:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ed9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.685333:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ed9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.685352:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ed9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.685359:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760698.685372:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ed9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.685380:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ed9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.685401:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ed9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.685409:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ed9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760698.685496:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495709440:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38490us (38718us total) trans 4294972256 rc 0/0 00000100:00100000:2.0:1789760698.685507:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4968 00000100:00100000:2.0:1789760698.685516:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682876739935 time 682967709569 state 0x2 ref 0 00000001:00080000:0.0:1789760698.687524:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972256 is committed 00000100:00100000:1.0:1789760698.695976:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760698.696094:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495709568 00000100:00100000:2.0:1789760698.696104:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495709568 00000100:00100000:2.0:1789760698.696157:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682876739935 time 682967709569 state 0x0 ref 1 00000100:00100000:2.0:1789760698.696173:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682967969985 time 682967709569 state 0x1 ref 1 00000100:00100000:2.0:1789760698.696240:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682967969985 time 683016956699 state 0x0 ref 1 00000100:00100000:2.0:1789760698.696254:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495709568:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760698.696493:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20234240, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:2.0:1789760698.696505:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ee0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.696521:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760698.696525:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760698.697160:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ee0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.697180:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ee0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760698.767845:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20234240 -> 20238336 00010000:00010000:2.0:1789760698.767866:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ee0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.767879:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ee0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.767888:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ee0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.767895:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760698.767900:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ee0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.767910:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ee0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.767928:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ee0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.767936:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ee0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760698.768236:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495709568:12345-192.168.203.9@tcp:4:dd.0 Request processed in 71978us (72263us total) trans 4294972257 rc 0/0 00000100:00100000:2.0:1789760698.768248:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4969 00000100:00100000:2.0:1789760698.768258:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682967969985 time 683016956699 state 0x2 ref 0 00000001:00080000:0.0:1789760698.771102:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972257 is committed 00000100:00100000:1.0:1789760698.783551:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760698.783753:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495709696 00000100:00100000:2.0:1789760698.783764:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495709696 00000100:00100000:2.0:1789760698.783818:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 682967969985 time 683016956699 state 0x0 ref 1 00000100:00100000:2.0:1789760698.783832:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683017217115 time 683016956699 state 0x1 ref 1 00000100:00100000:2.0:1789760698.783862:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683017217115 time 683104578613 state 0x0 ref 1 00000100:00100000:2.0:1789760698.783875:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495709696:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760698.783920:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20238336, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:2.0:1789760698.783929:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ee7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.783946:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760698.783951:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760698.783957:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ee7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.783971:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ee7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760698.854632:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972258 is committed 00002000:00010000:2.0:1789760698.854894:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20238336 -> 20242432 00010000:00010000:2.0:1789760698.854909:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ee7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.854922:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ee7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.855081:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ee7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.855091:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760698.855096:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ee7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.855106:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ee7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.855182:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ee7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.855190:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ee7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760698.855393:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495709696:12345-192.168.203.9@tcp:4:dd.0 Request processed in 71516us (71848us total) trans 4294972258 rc 0/0 00000100:00100000:2.0:1789760698.855407:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4970 00000100:00100000:2.0:1789760698.855836:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683017217115 time 683104578613 state 0x2 ref 0 00000100:00100000:0.0:1789760698.861272:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760698.861698:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495709824 00000100:00100000:2.0:1789760698.861708:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495709824 00000100:00100000:2.0:1789760698.861761:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683017217115 time 683104578613 state 0x0 ref 1 00000100:00100000:2.0:1789760698.861776:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683104839029 time 683104578613 state 0x1 ref 1 00000100:00100000:2.0:1789760698.861806:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683104839029 time 683182522756 state 0x0 ref 1 00000100:00100000:2.0:1789760698.861818:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495709824:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760698.861861:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20242432, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:2.0:1789760698.861870:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544eee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.861884:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760698.861888:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760698.861894:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544eee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.861906:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544eee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760698.908312:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20242432 -> 20246528 00010000:00010000:2.0:1789760698.908327:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544eee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.908350:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544eee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.908366:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544eee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.908375:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760698.908380:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544eee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.908390:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544eee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.908411:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544eee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.908418:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544eee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760698.908518:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495709824:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46698us (47250us total) trans 4294972259 rc 0/0 00000100:00100000:2.0:1789760698.908529:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4971 00000100:00100000:2.0:1789760698.908538:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683104839029 time 683182522756 state 0x2 ref 0 00000100:00100000:0.0:1789760698.912246:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760698.912391:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495709952 00000100:00100000:2.0:1789760698.912400:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495709952 00000100:00100000:2.0:1789760698.913352:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683104839029 time 683182522756 state 0x0 ref 1 00000100:00100000:2.0:1789760698.913378:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683182783172 time 683182522756 state 0x1 ref 1 00000100:00100000:2.0:1789760698.913458:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683182783172 time 683234173796 state 0x0 ref 1 00000100:00100000:2.0:1789760698.913474:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495709952:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760698.913542:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20246528, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:2.0:1789760698.913551:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ef5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.913567:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760698.913571:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760698.913578:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ef5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.913592:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ef5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760698.945460:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972260 is committed 00002000:00010000:2.0:1789760698.945728:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20246528 -> 20250624 00010000:00010000:2.0:1789760698.945744:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ef5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.945758:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ef5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.945773:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ef5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.945781:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760698.945787:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ef5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.945796:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ef5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.945816:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ef5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.945823:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544ef5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760698.945915:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495709952:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32440us (33671us total) trans 4294972260 rc 0/0 00000100:00100000:2.0:1789760698.945925:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4972 00000100:00100000:2.0:1789760698.945934:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683182783172 time 683234173796 state 0x2 ref 0 00000100:00100000:1.0:1789760698.951047:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760698.951233:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495710080 00000100:00100000:2.0:1789760698.951243:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495710080 00000100:00100000:2.0:1789760698.951334:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683182783172 time 683234173796 state 0x0 ref 1 00000100:00100000:2.0:1789760698.951366:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683234434212 time 683234173796 state 0x1 ref 1 00000100:00100000:2.0:1789760698.951388:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683234434212 time 683272104551 state 0x0 ref 1 00000100:00100000:2.0:1789760698.951399:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495710080:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760698.951444:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20250624, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:2.0:1789760698.951453:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544efc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.951468:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760698.951472:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760698.951479:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544efc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.951505:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544efc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760698.988472:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972261 is committed 00000100:00100000:0.0:1789760698.992630:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:2.0:1789760699.041796:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972262 is committed 00000100:00100000:2.0:1789760699.043922:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760699.044144:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495710336 00000100:00100000:0.0:1789760699.044153:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495710336 00000100:00100000:0.0:1789760699.044254:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683272364967 time 683313969544 state 0x0 ref 1 00000100:00100000:0.0:1789760699.044270:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683314229960 time 683313969544 state 0x1 ref 1 00000100:00100000:0.0:1789760699.044323:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683314229960 time 683365039260 state 0x0 ref 1 00000100:00100000:0.0:1789760699.044412:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495710336:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760699.044459:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20258816, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:0.0:1789760699.044468:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544f0a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.044484:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=683s v=(0 0) 00010000:00010000:0.0:1789760699.044498:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760699.044506:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544f0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.044520:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544f0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760699.102979:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20258816 -> 20262912 00010000:00010000:0.0:1789760699.102998:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544f0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.103011:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544f0a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.103029:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544f0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.103037:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760699.103043:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544f0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.103052:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544f0a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.103072:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544f0a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.103080:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354544f0a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760699.103180:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495710336:12345-192.168.203.9@tcp:4:dd.0 Request processed in 58841us (59260us total) trans 4294972263 rc 0/0 00000100:00100000:0.0:1789760699.103190:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4975 00000100:00100000:0.0:1789760699.103201:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683314229960 time 683365039260 state 0x2 ref 0 00000100:00100000:0.0:1789760699.108757:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00002000:00010000:2.0:1789760699.158801:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20262912 -> 20267008 00010000:00010000:2.0:1789760699.158821:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544f11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.158834:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544f11 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.158844:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544f11 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.158857:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760699.158862:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544f11 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.158872:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544f11 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.158887:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544f11 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.158892:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544f11 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760699.158992:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495710464:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49899us (50242us total) trans 4294972264 rc 0/0 00000100:00100000:2.0:1789760699.159000:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4976 00000100:00100000:2.0:1789760699.159053:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683365299676 time 683429796631 state 0x2 ref 0 00000100:00100000:2.0:1789760699.162997:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:1.0:1789760699.198949:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972265 is committed 00002000:00010000:2.0:1789760699.199157:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20267008 -> 20271104 00010000:00010000:2.0:1789760699.199173:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354544f18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.199186:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354544f18 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.199202:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354544f18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.199211:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760699.199216:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354544f18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.199226:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354544f18 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.199245:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354544f18 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.199254:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354544f18 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760699.199422:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495710592:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36168us (36429us total) trans 4294972265 rc 0/0 00000100:00100000:2.0:1789760699.199433:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4977 00000100:00100000:2.0:1789760699.199444:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683430057047 time 683483956220 state 0x2 ref 0 00000100:00100000:0.0:1789760699.211251:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760699.211455:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495710720 00000100:00100000:2.0:1789760699.211465:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495710720 00000100:00100000:2.0:1789760699.211528:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683430057047 time 683483956220 state 0x0 ref 1 00000100:00100000:2.0:1789760699.211545:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683484216636 time 683483956220 state 0x1 ref 1 00000100:00100000:2.0:1789760699.211576:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683484216636 time 683532292774 state 0x0 ref 1 00000100:00100000:2.0:1789760699.211588:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495710720:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760699.211639:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20271104, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:2.0:1789760699.211647:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f1f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.211662:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760699.211667:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760699.211673:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.211687:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.234033:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760699.269085:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20271104 -> 20275200 00010000:00010000:2.0:1789760699.269101:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.269115:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f1f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.269124:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f1f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.269131:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760699.269137:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f1f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.269146:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f1f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.269164:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f1f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.269170:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f1f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760699.269367:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495710720:12345-192.168.203.9@tcp:4:dd.0 Request processed in 57767us (58113us total) trans 4294972266 rc 0/0 00000100:00100000:2.0:1789760699.269378:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4978 00000100:00100000:2.0:1789760699.269389:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683484216636 time 683532292774 state 0x2 ref 0 00000100:00100000:1.0:1789760699.277447:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760699.277541:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495710848 00000100:00100000:2.0:1789760699.277551:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495710848 00000100:00100000:2.0:1789760699.277669:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683484216636 time 683532292774 state 0x0 ref 1 00000100:00100000:2.0:1789760699.277745:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683532553190 time 683532292774 state 0x1 ref 1 00000100:00100000:2.0:1789760699.277805:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683532553190 time 683598521564 state 0x0 ref 1 00000100:00100000:2.0:1789760699.277818:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495710848:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760699.277864:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20275200, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:2.0:1789760699.277873:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544f26 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.277889:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760699.277893:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760699.277899:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544f26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.277911:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544f26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760699.338956:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760699.340779:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495710976 00000100:00100000:2.0:1789760699.340790:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495710976 00000100:00100000:2.0:1789760699.340845:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683532553190 time 683598521564 state 0x0 ref 1 00000100:00100000:2.0:1789760699.340860:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683598781980 time 683598521564 state 0x1 ref 1 00000100:00100000:2.0:1789760699.340878:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683598781980 time 683661595518 state 0x0 ref 1 00000100:00100000:2.0:1789760699.340888:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495710976:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760699.340932:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20279296, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:2.0:1789760699.340940:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f2d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.340956:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760699.340961:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760699.340967:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.340982:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760699.401005:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20279296 -> 20283392 00010000:00010000:2.0:1789760699.401134:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.401148:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f2d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.401157:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.401165:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760699.401171:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.401180:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.401201:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f2d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.401209:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f2d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760699.401324:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495710976:12345-192.168.203.9@tcp:4:dd.0 Request processed in 60431us (62380us total) trans 4294972268 rc 0/0 00000100:00100000:2.0:1789760699.401341:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4980 00000100:00100000:2.0:1789760699.401352:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683598781980 time 683661595518 state 0x2 ref 0 00000100:00100000:0.0:1789760699.405953:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760699.406182:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495711104 00000100:00100000:2.0:1789760699.406192:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495711104 00000100:00100000:2.0:1789760699.406241:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683598781980 time 683661595518 state 0x0 ref 1 00000100:00100000:2.0:1789760699.406256:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683661855934 time 683661595518 state 0x1 ref 1 00000100:00100000:2.0:1789760699.406312:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683661855934 time 683727028923 state 0x0 ref 1 00000100:00100000:2.0:1789760699.406326:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495711104:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760699.406397:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20283392, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:2.0:1789760699.406407:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f34 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.406424:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760699.406429:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760699.406435:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.406448:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760699.444720:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20283392 -> 20287488 00010000:00010000:2.0:1789760699.444736:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.444749:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f34 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.444759:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.444767:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760699.444772:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.444781:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f34 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.444801:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f34 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.444809:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f34 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760699.444895:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495711104:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38568us (38948us total) trans 4294972269 rc 0/0 00000100:00100000:2.0:1789760699.444909:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4981 00000100:00100000:2.0:1789760699.444919:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683661855934 time 683727028923 state 0x2 ref 0 00000100:00100000:1.0:1789760699.449125:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:2.0:1789760699.510432:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972270 is committed 00000100:00100000:1.0:1789760699.514015:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760699.514117:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495711360 00000100:00100000:2.0:1789760699.514127:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495711360 00000100:00100000:2.0:1789760699.514178:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683727289339 time 683770044469 state 0x0 ref 1 00000100:00100000:2.0:1789760699.514193:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683770304885 time 683770044469 state 0x1 ref 1 00000100:00100000:2.0:1789760699.514255:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683770304885 time 683834971203 state 0x0 ref 1 00000100:00100000:2.0:1789760699.514268:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495711360:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760699.514342:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20291584, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:2.0:1789760699.514351:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f42 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.514376:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760699.514381:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760699.514387:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.514400:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760699.542743:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20291584 -> 20295680 00010000:00010000:2.0:1789760699.542765:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.542779:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f42 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.542787:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f42 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.542795:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760699.542800:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f42 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.542809:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f42 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.542827:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f42 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.542835:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f42 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760699.542933:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495711360:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28662us (28921us total) trans 4294972271 rc 0/0 00000100:00100000:2.0:1789760699.542944:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4983 00000100:00100000:2.0:1789760699.542954:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683770304885 time 683834971203 state 0x2 ref 0 00000100:00100000:0.0:1789760699.546771:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760699.546937:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495711488 00000100:00100000:2.0:1789760699.546947:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495711488 00000100:00100000:2.0:1789760699.546995:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683770304885 time 683834971203 state 0x0 ref 1 00000100:00100000:2.0:1789760699.547009:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683835231619 time 683834971203 state 0x1 ref 1 00000100:00100000:2.0:1789760699.547028:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683835231619 time 683867745752 state 0x0 ref 1 00000100:00100000:2.0:1789760699.547038:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903639d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495711488:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760699.547084:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20295680, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:2.0:1789760699.547093:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544f49 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.547110:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760699.547114:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760699.547120:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544f49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.547134:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544f49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760699.580097:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972272 is committed 00002000:00010000:1.0:1789760699.580256:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20295680 -> 20299776 00010000:00010000:1.0:1789760699.580391:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544f49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.580406:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544f49 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.580416:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544f49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.580423:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760699.580428:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544f49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.580437:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544f49 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.580457:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544f49 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.580465:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354544f49 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760699.580901:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903639d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495711488:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33857us (34127us total) trans 4294972272 rc 0/0 00000100:00100000:2.0:1789760699.580912:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4984 00000100:00100000:2.0:1789760699.580922:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683835231619 time 683867745752 state 0x2 ref 0 00000100:00100000:0.0:1789760699.585423:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760699.585939:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683835231619 time 683867745752 state 0x0 ref 1 00000100:00100000:2.0:1789760699.585954:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683868006168 time 683867745752 state 0x1 ref 1 00000100:00100000:2.0:1789760699.586012:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683868006168 time 683906727688 state 0x0 ref 1 00000100:00100000:2.0:1789760699.586025:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495711616:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760699.586073:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20299776, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:2.0:1789760699.586083:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f50 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.586098:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760699.586103:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760699.586110:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.586123:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760699.620769:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20299776 -> 20303872 00010000:00010000:1.0:1789760699.620788:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.620801:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f50 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.620811:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.620818:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760699.620823:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.620832:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f50 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.620850:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f50 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.620858:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544f50 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760699.620953:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495711616:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34925us (35535us total) trans 4294972273 rc 0/0 00000100:00100000:1.0:1789760699.620964:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4985 00000100:00100000:1.0:1789760699.620974:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683868006168 time 683906727688 state 0x2 ref 0 00000001:00080000:1.0:1789760699.623472:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972273 is committed 00000100:00100000:1.0:1789760699.626867:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760699.627502:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495711744 00000100:00100000:2.0:1789760699.627513:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495711744 00000100:00100000:2.0:1789760699.627567:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683868006168 time 683906727688 state 0x0 ref 1 00000100:00100000:2.0:1789760699.627582:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683906988104 time 683906727688 state 0x1 ref 1 00000100:00100000:2.0:1789760699.627781:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683906988104 time 683948497030 state 0x0 ref 1 00000100:00100000:2.0:1789760699.627796:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495711744:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760699.627842:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20303872, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:2.0:1789760699.627851:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544f57 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.627867:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760699.627871:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760699.627878:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544f57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.627891:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544f57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760699.681303:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520298752 00000100:00100000:1.0:1789760699.681311:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520298752 00000100:00100000:1.0:1789760699.681383:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 183 00000100:00100000:1.0:1789760699.681393:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02768e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520298752:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760699.681414:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528023552 pending 1703936 free 3837788160 avail 3817865216 00000100:00100000:1.0:1789760699.681460:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02768e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520298752:12345-0@lo:13:osp-pre-0-0.0 Request processed in 69us (298us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760699.681468:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 183 00000100:00100000:1.0:1789760699.681514:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f01835c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520298752:0@lo:13:osp-pre-0-0.0 00000001:00080000:1.0:1789760699.699097:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972274 is committed 00000100:00100000:2.0:1789760699.701361:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760699.701531:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495711872 00000100:00100000:0.0:1789760699.701540:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495711872 00000100:00100000:0.0:1789760699.701592:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683906988104 time 683948497030 state 0x0 ref 1 00000100:00100000:0.0:1789760699.701607:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683948757446 time 683948497030 state 0x1 ref 1 00000100:00100000:0.0:1789760699.701641:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 683948757446 time 684022358129 state 0x0 ref 1 00000100:00100000:0.0:1789760699.701653:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495711872:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760699.701712:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20307968, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa007 00010000:00010000:0.0:1789760699.701721:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354544f5e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.701736:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=684s v=(0 0) 00010000:00010000:0.0:1789760699.701741:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760699.701748:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354544f5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.701761:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354544f5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760699.749924:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972275 is committed 00000100:00100000:1.0:1789760699.753183:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760699.789778:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:1.0:1789760699.833799:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972277 is committed 00000100:00100000:1.0:1789760699.837809:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495712256 00000100:00100000:1.0:1789760699.837819:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495712256 00000100:00100000:1.0:1789760699.837873:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684074406119 time 684110685722 state 0x0 ref 1 00000100:00100000:1.0:1789760699.837887:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684110946138 time 684110685722 state 0x1 ref 1 00000100:00100000:1.0:1789760699.837918:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684110946138 time 684158636025 state 0x0 ref 1 00000100:00100000:1.0:1789760699.837930:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495712256:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760699.837979:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20320256, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa007 00010000:00010000:1.0:1789760699.837988:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544f73 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.838005:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=684s v=(0 0) 00010000:00010000:1.0:1789760699.838010:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760699.838018:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544f73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.838030:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544f73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760699.889347:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20320256 -> 20324352 00010000:00010000:1.0:1789760699.889373:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544f73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.889387:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544f73 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.889396:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544f73 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.889404:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760699.889510:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544f73 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.889521:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544f73 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.889541:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544f73 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.889549:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354544f73 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760699.889635:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495712256:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51701us (51984us total) trans 4294972278 rc 0/0 00000100:00100000:1.0:1789760699.889646:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4990 00000100:00100000:1.0:1789760699.889657:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684110946138 time 684158636025 state 0x2 ref 0 00000001:00080000:1.0:1789760699.889772:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972278 is committed 00000100:00100000:2.0:1789760699.899310:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760700.035960:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:2.0:1789760700.134126:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972280 is committed 00000100:00100000:1.0:1789760700.146107:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760700.146202:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495712640 00000100:00100000:2.0:1789760700.146211:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495712640 00000100:00100000:2.0:1789760700.146265:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684220584105 time 684357110928 state 0x0 ref 1 00000100:00100000:2.0:1789760700.146310:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684357371344 time 684357110928 state 0x1 ref 1 00000100:00100000:2.0:1789760700.146341:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684357371344 time 684467057369 state 0x0 ref 1 00000100:00100000:2.0:1789760700.146363:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495712640:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760700.146406:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20332544, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa007 00010000:00010000:2.0:1789760700.146415:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354544f88 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.146430:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=684s v=(0 0) 00010000:00010000:2.0:1789760700.146434:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760700.146440:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354544f88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.146462:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354544f88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760700.185041:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972281 is committed 00000100:00100000:2.0:1789760700.188142:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00002000:00010000:2.0:1789760700.232205:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20336640 -> 20340736 00010000:00010000:2.0:1789760700.232221:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354544f8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.232234:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354544f8f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.232244:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354544f8f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.232251:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760700.232256:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354544f8f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.232265:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354544f8f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.232410:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354544f8f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.232420:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354544f8f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760700.232545:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495712768:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43985us (44406us total) trans 4294972282 rc 0/0 00000100:00100000:2.0:1789760700.232556:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4994 00000100:00100000:2.0:1789760700.232565:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684467317785 time 684509258872 state 0x2 ref 0 00000100:00100000:2.0:1789760700.236204:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00010000:00010000:2.0:1789760700.254904:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760700.257482:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20340736 -> 20344832 00010000:00010000:2.0:1789760700.257504:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544f96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.257518:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544f96 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.257529:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544f96 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.257543:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760700.257549:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544f96 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.257559:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544f96 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.257584:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544f96 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.257592:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354544f96 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760700.258033:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495712896:12345-192.168.203.9@tcp:4:dd.0 Request processed in 21481us (21833us total) trans 4294972283 rc 0/0 00000100:00100000:2.0:1789760700.258044:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4995 00000100:00100000:2.0:1789760700.258055:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684509519288 time 684557253019 state 0x2 ref 0 00000100:00100000:1.0:1789760700.262274:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:2.0:1789760700.278135:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972284 is committed 00002000:00010000:0.0:1789760700.278273:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20344832 -> 20348928 00010000:00010000:0.0:1789760700.278323:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544f9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.278337:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544f9d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.278423:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544f9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.278439:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760700.278444:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544f9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.278453:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544f9d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.278589:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544f9d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.278600:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354544f9d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760700.278756:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495713024:12345-192.168.203.9@tcp:4:dd.0 Request processed in 16137us (16485us total) trans 4294972284 rc 0/0 00000100:00100000:0.0:1789760700.278767:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4996 00000100:00100000:0.0:1789760700.278778:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684557513435 time 684583323054 state 0x2 ref 0 00000100:00100000:1.0:1789760700.286452:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760700.286502:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495713152 00000100:00100000:2.0:1789760700.286512:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495713152 00000100:00100000:2.0:1789760700.286563:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684557513435 time 684583323054 state 0x0 ref 1 00000100:00100000:2.0:1789760700.286578:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684583583470 time 684583323054 state 0x1 ref 1 00000100:00100000:2.0:1789760700.286609:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684583583470 time 684607324963 state 0x0 ref 1 00000100:00100000:2.0:1789760700.286622:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495713152:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760700.286671:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20348928, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa007 00010000:00010000:2.0:1789760700.286680:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354544fa4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.286695:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=684s v=(0 0) 00010000:00010000:2.0:1789760700.286700:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760700.286706:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354544fa4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.286720:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354544fa4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760700.324869:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20348928 -> 20353024 00010000:00010000:1.0:1789760700.324885:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354544fa4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.324899:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354544fa4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.324909:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354544fa4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.324917:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760700.324921:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354544fa4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.324931:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354544fa4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.324956:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354544fa4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.324965:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354544fa4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760700.325056:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495713152:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38430us (38607us total) trans 4294972285 rc 0/0 00000100:00100000:1.0:1789760700.325067:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4997 00000100:00100000:1.0:1789760700.325076:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684583583470 time 684607324963 state 0x2 ref 0 00000100:00100000:2.0:1789760700.343825:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760700.343927:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495713280 00000100:00100000:0.0:1789760700.343937:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495713280 00000100:00100000:0.0:1789760700.343992:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684583583470 time 684607324963 state 0x0 ref 1 00000100:00100000:0.0:1789760700.344007:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684607585379 time 684607324963 state 0x1 ref 1 00000100:00100000:0.0:1789760700.344037:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684607585379 time 684664753741 state 0x0 ref 1 00000100:00100000:0.0:1789760700.344053:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495713280:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760700.344102:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20353024, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa007 00010000:00010000:0.0:1789760700.344111:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354544fab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.344127:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=684s v=(0 0) 00010000:00010000:0.0:1789760700.344131:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760700.344138:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354544fab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.344164:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354544fab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760700.383056:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972286 is committed 00002000:00010000:1.0:1789760700.383146:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20353024 -> 20357120 00010000:00010000:1.0:1789760700.383163:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354544fab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.383176:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354544fab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.383186:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354544fab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.383193:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ae00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760700.383198:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354544fab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.383213:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354544fab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.383234:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354544fab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.383242:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354544fab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760700.383565:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495713280:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39508us (39743us total) trans 4294972286 rc 0/0 00000100:00100000:1.0:1789760700.383577:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4998 00000100:00100000:1.0:1789760700.383586:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684607585379 time 684664753741 state 0x2 ref 0 00000100:00100000:0.0:1789760700.388127:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495713408 00000100:00100000:0.0:1789760700.388136:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495713408 00000100:00100000:0.0:1789760700.388195:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684607585379 time 684664753741 state 0x0 ref 1 00000100:00100000:0.0:1789760700.388210:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684665014157 time 684664753741 state 0x1 ref 1 00000100:00100000:0.0:1789760700.388273:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684665014157 time 684708989394 state 0x0 ref 1 00000100:00100000:0.0:1789760700.388305:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495713408:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760700.388370:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20357120, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa007 00010000:00010000:0.0:1789760700.388380:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354544fb2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.388403:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=684s v=(0 0) 00010000:00010000:0.0:1789760700.388408:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760700.388414:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354544fb2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.388428:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354544fb2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760700.460803:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972287 is committed 00002000:00010000:1.0:1789760700.461084:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20357120 -> 20361216 00010000:00010000:1.0:1789760700.461099:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354544fb2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.461112:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354544fb2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.461121:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354544fb2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.461129:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760700.461134:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354544fb2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.461144:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354544fb2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.461162:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354544fb2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.461171:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354544fb2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760700.461254:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495713408:12345-192.168.203.9@tcp:4:dd.0 Request processed in 72949us (73240us total) trans 4294972287 rc 0/0 00000100:00100000:1.0:1789760700.461265:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 4999 00000100:00100000:1.0:1789760700.461274:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684665014157 time 684708989394 state 0x2 ref 0 00000100:00100000:1.0:1789760700.467439:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760700.467516:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495713536 00000100:00100000:2.0:1789760700.467526:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495713536 00000100:00100000:2.0:1789760700.467581:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684665014157 time 684708989394 state 0x0 ref 1 00000100:00100000:2.0:1789760700.467595:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684709249810 time 684708989394 state 0x1 ref 1 00000100:00100000:2.0:1789760700.467634:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684709249810 time 684788342195 state 0x0 ref 1 00000100:00100000:2.0:1789760700.467646:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495713536:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760700.467690:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20361216, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa007 00010000:00010000:2.0:1789760700.467700:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544fb9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.467714:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=684s v=(0 0) 00010000:00010000:2.0:1789760700.467718:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760700.467723:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544fb9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.467736:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544fb9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760700.537202:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20361216 -> 20365312 00010000:00010000:1.0:1789760700.537219:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544fb9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.537245:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544fb9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.537254:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544fb9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.537266:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760700.537271:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544fb9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.537312:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544fb9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.537341:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544fb9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.537358:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354544fb9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760700.537702:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495713536:12345-192.168.203.9@tcp:4:dd.0 Request processed in 70051us (70275us total) trans 4294972288 rc 0/0 00000100:00100000:1.0:1789760700.537714:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5000 00000100:00100000:1.0:1789760700.537726:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684709249810 time 684788342195 state 0x2 ref 0 00000100:00100000:2.0:1789760700.540441:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760700.540654:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495713664 00000100:00100000:1.0:1789760700.540665:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495713664 00000100:00100000:1.0:1789760700.540720:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684709249810 time 684788342195 state 0x0 ref 1 00000100:00100000:1.0:1789760700.540735:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684788602611 time 684788342195 state 0x1 ref 1 00000100:00100000:1.0:1789760700.540805:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684788602611 time 684861521595 state 0x0 ref 1 00000100:00100000:1.0:1789760700.540822:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495713664:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760700.540873:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20365312, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa007 00010000:00010000:1.0:1789760700.540882:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354544fc0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.540898:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=684s v=(0 0) 00010000:00010000:1.0:1789760700.540903:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760700.540910:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354544fc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.540923:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354544fc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760700.561322:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972289 is committed 00002000:00010000:2.0:1789760700.561377:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20365312 -> 20369408 00010000:00010000:2.0:1789760700.561393:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354544fc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.561406:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354544fc0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.561413:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354544fc0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.561418:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760700.561421:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354544fc0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.561428:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354544fc0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.561444:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354544fc0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.561449:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354544fc0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760700.561539:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495713664:12345-192.168.203.9@tcp:4:dd.0 Request processed in 20716us (21105us total) trans 4294972289 rc 0/0 00000100:00100000:2.0:1789760700.561546:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5001 00000100:00100000:2.0:1789760700.561555:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684788602611 time 684861521595 state 0x2 ref 0 00000100:00100000:1.0:1789760700.569939:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495713792 00000100:00100000:1.0:1789760700.569949:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495713792 00000100:00100000:1.0:1789760700.570001:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684788602611 time 684861521595 state 0x0 ref 1 00000100:00100000:1.0:1789760700.570016:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684861782011 time 684861521595 state 0x1 ref 1 00000100:00100000:1.0:1789760700.570053:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684861782011 time 684890769230 state 0x0 ref 1 00000100:00100000:1.0:1789760700.570065:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495713792:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760700.570114:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20369408, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa007 00010000:00010000:1.0:1789760700.570123:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354544fc7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.570144:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=684s v=(0 0) 00010000:00010000:1.0:1789760700.570149:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760700.570155:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354544fc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.570168:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354544fc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760700.596490:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972290 is committed 00002000:00010000:2.0:1789760700.596602:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20369408 -> 20373504 00010000:00010000:2.0:1789760700.596620:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354544fc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.596634:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354544fc7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.596644:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354544fc7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.596652:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760700.596657:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354544fc7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.596672:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354544fc7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.596692:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354544fc7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.596701:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354544fc7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760700.596787:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495713792:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26720us (27542us total) trans 4294972290 rc 0/0 00000100:00100000:2.0:1789760700.596797:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5002 00000100:00100000:2.0:1789760700.596807:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684861782011 time 684890769230 state 0x2 ref 0 00000100:00100000:0.0:1789760700.599707:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760700.599806:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495713920 00000100:00100000:1.0:1789760700.599817:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495713920 00000100:00100000:1.0:1789760700.599873:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684861782011 time 684890769230 state 0x0 ref 1 00000100:00100000:1.0:1789760700.599887:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684891029646 time 684890769230 state 0x1 ref 1 00000100:00100000:1.0:1789760700.599953:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684891029646 time 684920668975 state 0x0 ref 1 00000100:00100000:1.0:1789760700.599966:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495713920:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760700.600027:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20373504, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa007 00010000:00010000:1.0:1789760700.600081:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354544fce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.600098:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=684s v=(0 0) 00010000:00010000:1.0:1789760700.600108:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760700.600115:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354544fce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.600133:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354544fce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760700.635390:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972291 is committed 00000100:00100000:2.0:1789760700.638187:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495714048 00000100:00100000:2.0:1789760700.638195:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495714048 00000100:00100000:2.0:1789760700.638240:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684891029646 time 684920668975 state 0x0 ref 1 00000100:00100000:2.0:1789760700.638251:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684920929391 time 684920668975 state 0x1 ref 1 00000100:00100000:2.0:1789760700.638276:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684920929391 time 684958993018 state 0x0 ref 1 00000100:00100000:2.0:1789760700.638445:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495714048:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760700.638490:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20377600, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa007 00010000:00010000:2.0:1789760700.638497:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544fd5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.638510:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=684s v=(0 0) 00010000:00010000:2.0:1789760700.638513:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760700.638517:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544fd5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.638527:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544fd5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760700.658694:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972292 is committed 00002000:00010000:2.0:1789760700.658716:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20377600 -> 20381696 00002000:00010000:2.0:1789760700.658721:0:6714:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760699 -> 1789760700 00002000:00010000:2.0:1789760700.658724:0:6714:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760699 -> 1789760700 00010000:00010000:2.0:1789760700.658737:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544fd5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.658750:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544fd5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.658759:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544fd5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.658766:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760700.658781:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544fd5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.658790:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544fd5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.658812:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544fd5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.658819:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544fd5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760700.658909:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495714048:12345-192.168.203.9@tcp:4:dd.0 Request processed in 20463us (20813us total) trans 4294972292 rc 0/0 00000100:00100000:2.0:1789760700.658920:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5004 00000100:00100000:2.0:1789760700.658929:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684920929391 time 684958993018 state 0x2 ref 0 00000100:00100000:1.0:1789760700.661683:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760700.661770:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495714176 00000100:00100000:2.0:1789760700.661780:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495714176 00000100:00100000:2.0:1789760700.661840:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684920929391 time 684958993018 state 0x0 ref 1 00000100:00100000:2.0:1789760700.661857:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684959253434 time 684958993018 state 0x1 ref 1 00000100:00100000:2.0:1789760700.661894:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684959253434 time 684982610969 state 0x0 ref 1 00000100:00100000:2.0:1789760700.661908:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495714176:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760700.661959:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20381696, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa007 00010000:00010000:2.0:1789760700.661967:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544fdc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.661983:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=684s v=(0 0) 00010000:00010000:2.0:1789760700.661988:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760700.661995:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544fdc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.662009:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544fdc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760700.682340:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20381696 -> 20385792 00010000:00010000:1.0:1789760700.682371:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544fdc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.682384:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544fdc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.682393:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544fdc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.682402:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760700.682407:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544fdc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.682417:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544fdc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.682439:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544fdc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.682447:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354544fdc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760700.682536:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495714176:12345-192.168.203.9@tcp:4:dd.0 Request processed in 20625us (20917us total) trans 4294972293 rc 0/0 00000100:00100000:1.0:1789760700.682546:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5005 00000100:00100000:1.0:1789760700.682556:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684959253434 time 684982610969 state 0x2 ref 0 00000100:00100000:0.0:1789760700.685910:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:1.0:1789760700.707313:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972294 is committed 00000100:00100000:2.0:1789760700.716830:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495714432 00000100:00100000:2.0:1789760700.716842:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495714432 00000100:00100000:2.0:1789760700.716897:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 684982871385 time 685006835646 state 0x0 ref 1 00000100:00100000:2.0:1789760700.716915:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685007096062 time 685006835646 state 0x1 ref 1 00000100:00100000:2.0:1789760700.716949:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685007096062 time 685037666162 state 0x0 ref 1 00000100:00100000:2.0:1789760700.716962:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495714432:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760700.717022:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20389888, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa007 00010000:00010000:2.0:1789760700.717031:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544fea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.717047:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=685s v=(0 0) 00010000:00010000:2.0:1789760700.717051:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760700.717057:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544fea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.717071:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354544fea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760700.773826:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972295 is committed 00000100:00100000:1.0:1789760700.780484:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760700.780741:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495714560 00000100:00100000:2.0:1789760700.780751:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495714560 00000100:00100000:2.0:1789760700.780820:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685007096062 time 685037666162 state 0x0 ref 1 00000100:00100000:2.0:1789760700.780838:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685037926578 time 685037666162 state 0x1 ref 1 00000100:00100000:2.0:1789760700.780868:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685037926578 time 685101585138 state 0x0 ref 1 00000100:00100000:2.0:1789760700.780880:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495714560:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760700.780935:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20393984, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa007 00010000:00010000:2.0:1789760700.780945:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544ff1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.780962:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=685s v=(0 0) 00010000:00010000:2.0:1789760700.780967:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760700.780973:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544ff1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.780987:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544ff1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760700.824248:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20393984 -> 20398080 00010000:00010000:2.0:1789760700.824262:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544ff1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.824276:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544ff1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.824306:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544ff1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.824313:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760700.824318:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544ff1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.824328:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544ff1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.824361:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544ff1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.824369:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544ff1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760700.824486:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495714560:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43605us (44009us total) trans 4294972296 rc 0/0 00000100:00100000:2.0:1789760700.824494:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5008 00000100:00100000:2.0:1789760700.824502:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685037926578 time 685101585138 state 0x2 ref 0 00000100:00100000:0.0:1789760700.828185:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00002000:00010000:0.0:1789760700.866961:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20398080 -> 20402176 00010000:00010000:0.0:1789760700.866977:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544ff8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.866991:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544ff8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.867000:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544ff8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.867007:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760700.867015:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544ff8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.867023:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544ff8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.867044:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544ff8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.867051:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354544ff8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760700.867936:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495714688:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39149us (39751us total) trans 4294972297 rc 0/0 00000100:00100000:0.0:1789760700.867946:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5009 00000100:00100000:0.0:1789760700.867956:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685101845554 time 685149484166 state 0x2 ref 0 00000100:00100000:2.0:1789760700.872069:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495714816 00000100:00100000:2.0:1789760700.872079:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495714816 00000100:00100000:2.0:1789760700.872138:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685101845554 time 685149484166 state 0x0 ref 1 00000100:00100000:2.0:1789760700.874561:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685149744582 time 685149484166 state 0x1 ref 1 00000100:00100000:2.0:1789760700.874786:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685149744582 time 685195501065 state 0x0 ref 1 00000100:00100000:2.0:1789760700.874812:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495714816:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760700.874865:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20402176, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa007 00010000:00010000:2.0:1789760700.874874:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544fff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.874891:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=685s v=(0 0) 00010000:00010000:2.0:1789760700.874895:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760700.874901:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544fff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.874915:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544fff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760700.963728:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20402176 -> 20406272 00010000:00010000:2.0:1789760700.963744:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544fff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.963757:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544fff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.963773:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544fff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.963781:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760700.963786:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544fff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.963795:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544fff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.963816:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544fff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.963824:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354544fff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760700.963917:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495714816:12345-192.168.203.9@tcp:4:dd.0 Request processed in 89114us (91955us total) trans 4294972298 rc 0/0 00000100:00100000:2.0:1789760700.963928:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5010 00000100:00100000:2.0:1789760700.963938:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685149744582 time 685195501065 state 0x2 ref 0 00000100:00100000:1.0:1789760700.966757:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760700.966906:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495714944 00000100:00100000:2.0:1789760700.966916:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495714944 00000100:00100000:2.0:1789760700.966970:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685149744582 time 685195501065 state 0x0 ref 1 00000100:00100000:2.0:1789760700.966985:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685195761481 time 685195501065 state 0x1 ref 1 00000100:00100000:2.0:1789760700.967009:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685195761481 time 685287726731 state 0x0 ref 1 00000100:00100000:2.0:1789760700.967021:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495714944:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760700.967068:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20406272, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa007 00010000:00010000:2.0:1789760700.967078:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545006 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.967094:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=685s v=(0 0) 00010000:00010000:2.0:1789760700.967099:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760700.967105:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545006 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.967119:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545006 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760701.029654:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972299 is committed 00000100:00100000:0.0:1789760701.033761:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495715072 00000100:00100000:0.0:1789760701.033772:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495715072 00000100:00100000:0.0:1789760701.033836:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685195761481 time 685287726731 state 0x0 ref 1 00000100:00100000:0.0:1789760701.033852:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685287987147 time 685287726731 state 0x1 ref 1 00000100:00100000:0.0:1789760701.033877:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685287987147 time 685354593580 state 0x0 ref 1 00000100:00100000:0.0:1789760701.033890:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495715072:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760701.033938:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20410368, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa007 00010000:00010000:0.0:1789760701.033945:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454500d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.033958:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=685s v=(0 0) 00010000:00010000:0.0:1789760701.033962:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760701.033966:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454500d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.033976:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454500d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760701.101152:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972300 is committed 00002000:00010000:1.0:1789760701.101647:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20410368 -> 20414464 00010000:00010000:1.0:1789760701.101664:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454500d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.101676:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454500d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.101685:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454500d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.101691:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760701.101696:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454500d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.101705:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454500d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.101728:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454500d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.101735:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454500d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760701.101870:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495715072:12345-192.168.203.9@tcp:4:dd.0 Request processed in 67978us (68254us total) trans 4294972300 rc 0/0 00000100:00100000:1.0:1789760701.101880:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5012 00000100:00100000:1.0:1789760701.101889:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685287987147 time 685354593580 state 0x2 ref 0 00000100:00100000:2.0:1789760701.108835:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495715200 00000100:00100000:2.0:1789760701.108844:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495715200 00000100:00100000:2.0:1789760701.109089:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685287987147 time 685354593580 state 0x0 ref 1 00000100:00100000:2.0:1789760701.109107:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685354853996 time 685354593580 state 0x1 ref 1 00000100:00100000:2.0:1789760701.109183:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685354853996 time 685429898990 state 0x0 ref 1 00000100:00100000:2.0:1789760701.109197:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495715200:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760701.109257:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20414464, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa007 00010000:00010000:2.0:1789760701.109267:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545014 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.109311:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=685s v=(0 0) 00010000:00010000:2.0:1789760701.109316:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760701.109331:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545014 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.109353:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545014 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760701.157968:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972301 is committed 00002000:00010000:0.0:1789760701.158073:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20414464 -> 20418560 00010000:00010000:0.0:1789760701.158090:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545014 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.158105:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545014 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.158116:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545014 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.158123:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760701.158129:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545014 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.158143:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545014 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.158163:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545014 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.158171:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545014 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760701.158338:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495715200:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49139us (49601us total) trans 4294972301 rc 0/0 00000100:00100000:0.0:1789760701.158357:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5013 00000100:00100000:0.0:1789760701.158367:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685354853996 time 685429898990 state 0x2 ref 0 00000100:00100000:1.0:1789760701.162148:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495715328 00000100:00100000:1.0:1789760701.162158:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495715328 00000100:00100000:1.0:1789760701.162216:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685354853996 time 685429898990 state 0x0 ref 1 00000100:00100000:1.0:1789760701.162232:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685430159406 time 685429898990 state 0x1 ref 1 00000100:00100000:1.0:1789760701.162266:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685430159406 time 685482982199 state 0x0 ref 1 00000100:00100000:1.0:1789760701.162559:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495715328:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760701.162610:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20418560, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa007 00010000:00010000:1.0:1789760701.162619:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x493674835454501b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.162641:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=685s v=(0 0) 00010000:00010000:1.0:1789760701.162646:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760701.162652:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x493674835454501b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.162665:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x493674835454501b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760701.204621:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972302 is committed 00000100:00100000:1.0:1789760701.207808:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495715456 00000100:00100000:1.0:1789760701.207817:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495715456 00000100:00100000:1.0:1789760701.207863:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685430159406 time 685482982199 state 0x0 ref 1 00000100:00100000:1.0:1789760701.207875:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685483242615 time 685482982199 state 0x1 ref 1 00000100:00100000:1.0:1789760701.207898:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685483242615 time 685528616293 state 0x0 ref 1 00000100:00100000:1.0:1789760701.207906:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495715456:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760701.207943:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20422656, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa007 00010000:00010000:1.0:1789760701.207951:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545022 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.207966:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=685s v=(0 0) 00010000:00010000:1.0:1789760701.207971:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760701.207978:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545022 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.207996:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545022 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760701.230762:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972303 is committed 00002000:00010000:0.0:1789760701.230861:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20422656 -> 20426752 00010000:00010000:0.0:1789760701.230876:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545022 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.230889:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545022 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.230899:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545022 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.230906:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760701.230912:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545022 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.230922:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545022 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.230942:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545022 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.230950:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545022 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760701.231036:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495715456:12345-192.168.203.9@tcp:4:dd.0 Request processed in 23125us (23314us total) trans 4294972303 rc 0/0 00000100:00100000:0.0:1789760701.231045:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5015 00000100:00100000:0.0:1789760701.231055:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685483242615 time 685528616293 state 0x2 ref 0 00000100:00100000:2.0:1789760701.235312:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760701.235378:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495715584 00000100:00100000:0.0:1789760701.235387:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495715584 00000100:00100000:0.0:1789760701.235504:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685483242615 time 685528616293 state 0x0 ref 1 00000100:00100000:0.0:1789760701.235519:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685528876709 time 685528616293 state 0x1 ref 1 00000100:00100000:0.0:1789760701.235593:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685528876709 time 685556308974 state 0x0 ref 1 00000100:00100000:0.0:1789760701.235605:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495715584:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760701.235651:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20426752, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa007 00010000:00010000:0.0:1789760701.235661:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545029 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.235675:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=685s v=(0 0) 00010000:00010000:0.0:1789760701.235679:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760701.235686:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545029 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.235700:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545029 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.281150:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760701.299657:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972304 is committed 00002000:00010000:0.0:1789760701.299836:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20426752 -> 20430848 00010000:00010000:0.0:1789760701.299851:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545029 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.299864:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545029 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.299873:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545029 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.299881:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760701.299994:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545029 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.300005:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545029 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.300033:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545029 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.300042:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545029 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760701.300137:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495715584:12345-192.168.203.9@tcp:4:dd.0 Request processed in 64528us (64851us total) trans 4294972304 rc 0/0 00000100:00100000:0.0:1789760701.300147:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5016 00000100:00100000:0.0:1789760701.300157:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685528876709 time 685556308974 state 0x2 ref 0 00000100:00100000:2.0:1789760701.309192:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00002000:00010000:1.0:1789760701.347649:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20430848 -> 20434944 00010000:00010000:1.0:1789760701.347668:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545030 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.347691:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545030 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.347700:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545030 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.347708:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760701.347713:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545030 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.347722:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545030 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.347743:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545030 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.347751:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545030 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760701.347848:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495715712:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38427us (38658us total) trans 4294972305 rc 0/0 00000100:00100000:1.0:1789760701.347859:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5017 00000100:00100000:1.0:1789760701.347870:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685556569390 time 685630122730 state 0x2 ref 0 00000001:00080000:1.0:1789760701.347913:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972305 is committed 00000100:00100000:1.0:1789760701.355652:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760701.356009:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495715840 00000100:00100000:0.0:1789760701.356019:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495715840 00000100:00100000:0.0:1789760701.356077:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685556569390 time 685630122730 state 0x0 ref 1 00000100:00100000:0.0:1789760701.356093:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685630383146 time 685630122730 state 0x1 ref 1 00000100:00100000:0.0:1789760701.356161:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685630383146 time 685676877298 state 0x0 ref 1 00000100:00100000:0.0:1789760701.356174:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495715840:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760701.356224:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20434944, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa007 00010000:00010000:0.0:1789760701.356234:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545037 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.356251:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=685s v=(0 0) 00010000:00010000:0.0:1789760701.356256:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760701.356263:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545037 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.356276:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545037 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760701.391566:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20434944 -> 20439040 00010000:00010000:0.0:1789760701.391581:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545037 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.391593:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545037 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.391601:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545037 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.391609:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760701.391614:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545037 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.391622:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545037 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.391641:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545037 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.391653:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545037 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760701.391759:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495715840:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35583us (36113us total) trans 4294972306 rc 0/0 00000100:00100000:0.0:1789760701.391770:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5018 00000100:00100000:0.0:1789760701.391779:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685630383146 time 685676877298 state 0x2 ref 0 00000100:00100000:0.0:1789760701.395545:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495715968 00000100:00100000:0.0:1789760701.395556:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495715968 00000100:00100000:0.0:1789760701.395613:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685630383146 time 685676877298 state 0x0 ref 1 00000100:00100000:0.0:1789760701.395628:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685677137714 time 685676877298 state 0x1 ref 1 00000100:00100000:0.0:1789760701.395650:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685677137714 time 685716367401 state 0x0 ref 1 00000100:00100000:0.0:1789760701.395677:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04577b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495715968:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760701.395724:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20439040, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa007 00010000:00010000:0.0:1789760701.395733:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454503e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.395749:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=685s v=(0 0) 00010000:00010000:0.0:1789760701.395754:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760701.395760:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454503e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.395780:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454503e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760701.444499:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20439040 -> 20443136 00010000:00010000:1.0:1789760701.444589:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454503e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.444602:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454503e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.444612:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454503e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.444620:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760701.444625:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454503e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.444633:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454503e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.444652:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454503e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.445167:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454503e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760701.446968:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04577b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495715968:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49616us (50477us total) trans 4294972307 rc 0/0 00000100:00100000:1.0:1789760701.446983:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5019 00000100:00100000:1.0:1789760701.446994:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685677137714 time 685716367401 state 0x2 ref 0 00000100:00100000:1.0:1789760701.453469:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:0.0:1789760701.520483:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972308 is committed 00000100:00100000:0.0:1789760701.538495:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495716224 00000100:00100000:0.0:1789760701.538504:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495716224 00000100:00100000:0.0:1789760701.538569:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685716627817 time 685774544100 state 0x0 ref 1 00000100:00100000:0.0:1789760701.538586:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685774804516 time 685774544100 state 0x1 ref 1 00000100:00100000:0.0:1789760701.538614:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685774804516 time 685859330281 state 0x0 ref 1 00000100:00100000:0.0:1789760701.538627:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495716224:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760701.538674:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20447232, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa007 00010000:00010000:0.0:1789760701.538683:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454504c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.538698:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=685s v=(0 0) 00010000:00010000:0.0:1789760701.538702:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760701.538709:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454504c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.538723:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454504c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760701.585760:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972309 is committed 00000100:00100000:1.0:1789760701.592546:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495716352 00000100:00100000:1.0:1789760701.592557:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495716352 00000100:00100000:1.0:1789760701.592615:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685774804516 time 685859330281 state 0x0 ref 1 00000100:00100000:1.0:1789760701.592630:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685859590697 time 685859330281 state 0x1 ref 1 00000100:00100000:1.0:1789760701.592662:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685859590697 time 685913378808 state 0x0 ref 1 00000100:00100000:1.0:1789760701.592675:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495716352:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760701.592725:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20451328, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa007 00010000:00010000:1.0:1789760701.592736:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545053 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.592755:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=685s v=(0 0) 00010000:00010000:1.0:1789760701.592760:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760701.592768:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545053 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.592782:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545053 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760701.630269:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972310 is committed 00002000:00010000:1.0:1789760701.630437:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20451328 -> 20455424 00010000:00010000:1.0:1789760701.630453:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545053 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.630466:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545053 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.630473:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545053 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.630480:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760701.630485:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545053 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.630493:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545053 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.630517:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545053 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.630524:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545053 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760701.630616:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495716352:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37939us (38290us total) trans 4294972310 rc 0/0 00000100:00100000:1.0:1789760701.630626:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5022 00000100:00100000:1.0:1789760701.630636:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685859590697 time 685913378808 state 0x2 ref 0 00000100:00100000:1.0:1789760701.638556:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:2.0:1789760701.711084:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972311 is committed 00002000:00010000:1.0:1789760701.711185:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20455424 -> 20459520 00002000:00010000:1.0:1789760701.711191:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760700 -> 1789760701 00002000:00010000:1.0:1789760701.711194:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760700 -> 1789760701 00010000:00010000:1.0:1789760701.711207:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454505a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.711220:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454505a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.711230:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454505a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.711237:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760701.711242:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454505a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.711252:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454505a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.711433:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454505a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.711443:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454505a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760701.711534:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495716480:12345-192.168.203.9@tcp:4:dd.0 Request processed in 72602us (72984us total) trans 4294972311 rc 0/0 00000100:00100000:1.0:1789760701.711544:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5023 00000100:00100000:1.0:1789760701.711554:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685913639224 time 685959633497 state 0x2 ref 0 00000100:00100000:2.0:1789760701.720196:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760701.728705:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033b1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:59c4e264-9b75-482b-8813-3d1dc124f384:3317:1876699520298880:0@lo:400:kworker.0 00000100:00100000:1.0:1789760701.728775:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760701.728820:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3317:1876699520299392:0@lo:400:kworker.0 00000100:00100000:1.0:1789760701.728847:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760701.728872:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1789760701.728878:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3317:1876699520299008:0@lo:400:kworker.0 00000100:00100000:1.0:1789760701.728899:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760701.728916:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3317:1876699520299520:0@lo:400:kworker.0 00000100:00100000:1.0:1789760701.728937:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760701.728966:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520299136:0@lo:400:kworker.0 00000100:00100000:1.0:1789760701.728991:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760701.729007:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760701.729012:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520299264:0@lo:400:kworker.0 00000100:00100000:1.0:1789760701.729034:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760701.729086:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520298880 00000100:00100000:1.0:1789760701.729093:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520298880 00000100:00100000:1.0:1789760701.729135:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 385 00000100:00100000:1.0:1789760701.729143:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929012315850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:59c4e264-9b75-482b-8813-3d1dc124f384+19:3317:x1876699520298880:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760701.729192:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929012315850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:59c4e264-9b75-482b-8813-3d1dc124f384+19:3317:x1876699520298880:12345-0@lo:400:kworker.0 Request processed in 49us (417us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760701.729199:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 385 00000100:00100000:1.0:1789760701.729232:0:12710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520299392 00000100:00100000:1.0:1789760701.729237:0:12710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520299392 00000100:00100000:1.0:1789760701.729269:0:12710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520299008 00000100:00100000:1.0:1789760701.729272:0:12710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520299008 00000100:00100000:1.0:1789760701.729330:0:12710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520299136 00000100:00100000:1.0:1789760701.729333:0:12710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520299136 00000100:00100000:1.0:1789760701.729360:0:12710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 442 00000100:00100000:1.0:1789760701.729366:0:12710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1876699520299392:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760701.729400:0:12710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1876699520299392:12345-0@lo:400:kworker.0 Request processed in 33us (552us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760701.729407:0:12710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 442 00000100:00100000:1.0:1789760701.729419:0:12710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 443 00000100:00100000:1.0:1789760701.729425:0:12710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1876699520299008:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760701.729453:0:12710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1876699520299008:12345-0@lo:400:kworker.0 Request processed in 28us (553us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760701.729458:0:12710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 443 00000100:00100000:1.0:1789760701.729482:0:12710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 444 00000100:00100000:1.0:1789760701.729488:0:12710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036e56a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520299136:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760701.729519:0:12710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036e56a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520299136:12345-0@lo:400:kworker.0 Request processed in 31us (527us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760701.729526:0:12710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 444 00000100:00100000:1.0:1789760701.729549:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520299520 00000100:00100000:1.0:1789760701.729552:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520299520 00000100:00100000:1.0:1789760701.729576:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520299264 00000100:00100000:1.0:1789760701.729579:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520299264 00000100:00100000:1.0:1789760701.729599:0:6709:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 208 00000100:00100000:1.0:1789760701.729605:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3317:x1876699520299520:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760701.729637:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3317:x1876699520299520:12345-0@lo:400:kworker.0 Request processed in 32us (698us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760701.729642:0:6709:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 208 00000100:00100000:1.0:1789760701.729654:0:6709:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 209 00000100:00100000:1.0:1789760701.729660:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036e56300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520299264:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760701.729686:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036e56300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520299264:12345-0@lo:400:kworker.0 Request processed in 26us (651us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760701.729692:0:6709:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 209 00000100:00100000:1.0:1789760701.729734:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3317:1876699520299008:0@lo:400:kworker.0 00000100:00100000:1.0:1789760701.729744:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3317:1876699520299520:0@lo:400:kworker.0 00000100:00100000:1.0:1789760701.729751:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033b1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:59c4e264-9b75-482b-8813-3d1dc124f384:3317:1876699520298880:0@lo:400:kworker.0 00000100:00100000:1.0:1789760701.729760:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3317:1876699520299392:0@lo:400:kworker.0 00000100:00080000:1.0:1789760701.729787:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789760701.729791:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789760701.729793:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760701.729814:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520299264:0@lo:400:kworker.0 00000100:00100000:1.0:1789760701.729823:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520299136:0@lo:400:kworker.0 00000001:00080000:2.0:1789760701.813984:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972312 is committed 00002000:00010000:1.0:1789760701.814159:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20459520 -> 20463616 00010000:00010000:1.0:1789760701.814175:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545061 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.814188:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545061 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.814197:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545061 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.814204:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760701.814209:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545061 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.814218:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545061 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.814238:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545061 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.814246:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545061 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760701.814384:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495716608:12345-192.168.203.9@tcp:4:dd.0 Request processed in 93937us (94191us total) trans 4294972312 rc 0/0 00000100:00100000:1.0:1789760701.814394:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5024 00000100:00100000:1.0:1789760701.814405:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685959893913 time 686041146972 state 0x2 ref 0 00000100:00100000:2.0:1789760701.823574:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760701.823681:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495716736 00000100:00100000:1.0:1789760701.823691:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495716736 00000100:00100000:1.0:1789760701.823746:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 685959893913 time 686041146972 state 0x0 ref 1 00000100:00100000:1.0:1789760701.823761:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686041407388 time 686041146972 state 0x1 ref 1 00000100:00100000:1.0:1789760701.823794:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686041407388 time 686144511054 state 0x0 ref 1 00000100:00100000:1.0:1789760701.823805:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495716736:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760701.823854:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20463616, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa007 00010000:00010000:1.0:1789760701.823863:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545068 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.823878:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=686s v=(0 0) 00010000:00010000:1.0:1789760701.823882:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760701.823888:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545068 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.823902:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545068 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760701.876998:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972313 is committed 00002000:00010000:1.0:1789760701.877129:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20463616 -> 20467712 00010000:00010000:1.0:1789760701.877146:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545068 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.877166:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545068 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.877176:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545068 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.877184:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760701.877189:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545068 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.877198:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545068 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.877221:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545068 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.877229:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545068 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760701.877391:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495716736:12345-192.168.203.9@tcp:4:dd.0 Request processed in 53582us (53821us total) trans 4294972313 rc 0/0 00000100:00100000:1.0:1789760701.877403:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5025 00000100:00100000:1.0:1789760701.877414:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686041407388 time 686144511054 state 0x2 ref 0 00000100:00100000:2.0:1789760701.882766:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760701.882920:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495716864 00000100:00100000:1.0:1789760701.882930:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495716864 00000100:00100000:1.0:1789760701.882984:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686041407388 time 686144511054 state 0x0 ref 1 00000100:00100000:1.0:1789760701.883000:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686144771470 time 686144511054 state 0x1 ref 1 00000100:00100000:1.0:1789760701.883035:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686144771470 time 686203751302 state 0x0 ref 1 00000100:00100000:1.0:1789760701.883047:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495716864:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760701.883096:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20467712, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa007 00010000:00010000:1.0:1789760701.883106:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454506f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.883123:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=686s v=(0 0) 00010000:00010000:1.0:1789760701.883127:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760701.883134:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454506f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.883147:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454506f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760702.006245:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972314 is committed 00002000:00010000:1.0:1789760702.006842:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20467712 -> 20471808 00010000:00010000:1.0:1789760702.006859:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454506f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.006872:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454506f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.006886:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454506f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.006893:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760702.006899:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454506f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.006907:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454506f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.006927:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454506f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.006935:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454506f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.007030:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495716864:12345-192.168.203.9@tcp:4:dd.0 Request processed in 123980us (124269us total) trans 4294972314 rc 0/0 00000100:00100000:1.0:1789760702.007039:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5026 00000100:00100000:1.0:1789760702.007049:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686144771470 time 686203751302 state 0x2 ref 0 00000100:00100000:1.0:1789760702.014721:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:2.0:1789760702.095347:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972315 is committed 00002000:00010000:1.0:1789760702.095458:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20471808 -> 20475904 00010000:00010000:1.0:1789760702.095472:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545076 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.095491:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545076 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.095499:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545076 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.095507:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760702.095512:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545076 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.095521:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545076 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.095541:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545076 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.095548:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545076 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.095644:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495716992:12345-192.168.203.9@tcp:4:dd.0 Request processed in 80687us (80928us total) trans 4294972315 rc 0/0 00000100:00100000:1.0:1789760702.095655:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5027 00000100:00100000:1.0:1789760702.095730:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686204011718 time 686335660020 state 0x2 ref 0 00000100:00100000:2.0:1789760702.099528:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760702.099613:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495717120 00000100:00100000:1.0:1789760702.099624:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495717120 00000100:00100000:1.0:1789760702.099680:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686204011718 time 686335660020 state 0x0 ref 1 00000100:00100000:1.0:1789760702.099695:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686335920436 time 686335660020 state 0x1 ref 1 00000100:00100000:1.0:1789760702.099725:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686335920436 time 686420442617 state 0x0 ref 1 00000100:00100000:1.0:1789760702.099737:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495717120:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760702.099784:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20475904, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa007 00010000:00010000:1.0:1789760702.099793:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454507d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.099808:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=686s v=(0 0) 00010000:00010000:1.0:1789760702.099813:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.099819:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454507d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.099832:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454507d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760702.129107:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20475904 -> 20480000 00010000:00010000:2.0:1789760702.129122:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454507d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.129140:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454507d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.129150:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454507d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.129157:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760702.129162:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454507d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.129171:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454507d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.129191:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454507d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.129199:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454507d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760702.129375:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495717120:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29635us (29851us total) trans 4294972316 rc 0/0 00000100:00100000:2.0:1789760702.129386:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5028 00000100:00100000:2.0:1789760702.129395:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686335920436 time 686420442617 state 0x2 ref 0 00000100:00100000:2.0:1789760702.134239:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760702.134388:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495717248 00000100:00100000:1.0:1789760702.134397:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495717248 00000100:00100000:1.0:1789760702.134452:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686335920436 time 686420442617 state 0x0 ref 1 00000100:00100000:1.0:1789760702.134467:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686420703033 time 686420442617 state 0x1 ref 1 00000100:00100000:1.0:1789760702.134501:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686420703033 time 686455217184 state 0x0 ref 1 00000100:00100000:1.0:1789760702.134516:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495717248:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760702.134564:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20480000, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa007 00010000:00010000:1.0:1789760702.134572:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545084 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.134584:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=686s v=(0 0) 00010000:00010000:1.0:1789760702.134587:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.134591:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545084 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.134602:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545084 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760702.238663:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9290404af480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520299648:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760702.238733:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760702.239195:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9290404af480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520299648:0@lo:13:osp-pre-1-0.0 00002000:00010000:1.0:1789760702.286196:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20480000 -> 20484096 00010000:00010000:1.0:1789760702.286216:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545084 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.286230:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545084 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.286241:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545084 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.286249:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760702.286253:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545084 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.286264:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545084 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.286343:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545084 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.286352:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545084 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.286443:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495717248:12345-192.168.203.9@tcp:4:dd.0 Request processed in 151924us (152209us total) trans 4294972317 rc 0/0 00000100:00100000:1.0:1789760702.286454:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5029 00000100:00100000:1.0:1789760702.286465:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686420703033 time 686455217184 state 0x2 ref 0 00000100:00100000:1.0:1789760702.290545:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00010000:00010000:1.0:1789760702.302940:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760702.347037:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972318 is committed 00002000:00010000:1.0:1789760702.347163:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20484096 -> 20488192 00010000:00010000:1.0:1789760702.347178:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454508b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.347189:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454508b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.347197:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454508b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.347209:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760702.347214:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454508b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.347222:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454508b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.347240:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454508b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.347247:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454508b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.347545:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495717376:12345-192.168.203.9@tcp:4:dd.0 Request processed in 56768us (57004us total) trans 4294972318 rc 0/0 00000100:00100000:1.0:1789760702.347556:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5030 00000100:00100000:1.0:1789760702.347566:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686455477600 time 686611476710 state 0x2 ref 0 00000100:00100000:0.0:1789760702.359205:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760702.359341:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495717504 00000100:00100000:2.0:1789760702.359351:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495717504 00000100:00100000:2.0:1789760702.359404:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686455477600 time 686611476710 state 0x0 ref 1 00000100:00100000:2.0:1789760702.359419:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686611737126 time 686611476710 state 0x1 ref 1 00000100:00100000:2.0:1789760702.359440:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686611737126 time 686680158153 state 0x0 ref 1 00000100:00100000:2.0:1789760702.359452:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495717504:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760702.359501:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20488192, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa007 00010000:00010000:2.0:1789760702.359510:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545092 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.359525:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=686s v=(0 0) 00010000:00010000:2.0:1789760702.359529:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760702.359535:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545092 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.359548:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545092 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760702.489935:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972319 is committed 00002000:00010000:1.0:1789760702.490141:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20488192 -> 20492288 00010000:00010000:1.0:1789760702.490158:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545092 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.490172:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545092 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.490186:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545092 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.490195:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760702.490200:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545092 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.490210:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545092 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.490229:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545092 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.490238:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545092 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.490523:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495717504:12345-192.168.203.9@tcp:4:dd.0 Request processed in 131068us (131321us total) trans 4294972319 rc 0/0 00000100:00100000:1.0:1789760702.490534:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5031 00000100:00100000:1.0:1789760702.490544:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686611737126 time 686680158153 state 0x2 ref 0 00000100:00100000:1.0:1789760702.494810:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495717632 00000100:00100000:1.0:1789760702.494820:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495717632 00000100:00100000:1.0:1789760702.494889:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686611737126 time 686680158153 state 0x0 ref 1 00000100:00100000:1.0:1789760702.494905:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686680418569 time 686680158153 state 0x1 ref 1 00000100:00100000:1.0:1789760702.494979:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686680418569 time 686815695562 state 0x0 ref 1 00000100:00100000:1.0:1789760702.494992:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495717632:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760702.495037:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20492288, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa007 00010000:00010000:1.0:1789760702.495047:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545099 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.495064:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=686s v=(0 0) 00010000:00010000:1.0:1789760702.495069:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.495075:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545099 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.495089:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545099 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760702.588508:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972320 is committed 00002000:00010000:1.0:1789760702.588573:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20492288 -> 20496384 00010000:00010000:1.0:1789760702.588589:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545099 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.588601:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545099 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.588610:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545099 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.588622:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760702.588627:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545099 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.588636:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545099 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.588726:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545099 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.588735:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545099 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.588829:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495717632:12345-192.168.203.9@tcp:4:dd.0 Request processed in 93834us (94352us total) trans 4294972320 rc 0/0 00000100:00100000:1.0:1789760702.588839:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5032 00000100:00100000:1.0:1789760702.588850:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686680418569 time 686815695562 state 0x2 ref 0 00000100:00100000:1.0:1789760702.597619:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:1.0:1789760702.694394:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972321 is committed 00000100:00100000:0.0:1789760702.698531:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760702.698622:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495717888 00000100:00100000:1.0:1789760702.698632:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495717888 00000100:00100000:1.0:1789760702.698756:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686815955978 time 686918569322 state 0x0 ref 1 00000100:00100000:1.0:1789760702.698771:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686918829738 time 686918569322 state 0x1 ref 1 00000100:00100000:1.0:1789760702.698807:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686918829738 time 687019523764 state 0x0 ref 1 00000100:00100000:1.0:1789760702.698820:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495717888:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760702.698866:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20500480, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa007 00010000:00010000:1.0:1789760702.698875:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545450a7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.698889:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=687s v=(0 0) 00010000:00010000:1.0:1789760702.698893:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.698899:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545450a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.698912:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545450a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760702.810136:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972322 is committed 00002000:00010000:1.0:1789760702.810250:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20500480 -> 20504576 00002000:00010000:1.0:1789760702.810255:0:6714:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760701 -> 1789760702 00002000:00010000:1.0:1789760702.810257:0:6714:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760701 -> 1789760702 00010000:00010000:1.0:1789760702.810270:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545450a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.810325:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545450a7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.810344:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545450a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.810352:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760702.810357:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545450a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.810366:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545450a7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.810385:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545450a7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.810392:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545450a7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.810479:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495717888:12345-192.168.203.9@tcp:4:dd.0 Request processed in 111657us (111951us total) trans 4294972322 rc 0/0 00000100:00100000:1.0:1789760702.810489:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5034 00000100:00100000:1.0:1789760702.810500:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686918829738 time 687019523764 state 0x2 ref 0 00000100:00100000:2.0:1789760702.817575:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760702.817668:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495718016 00000100:00100000:1.0:1789760702.817676:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495718016 00000100:00100000:1.0:1789760702.817733:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 686918829738 time 687019523764 state 0x0 ref 1 00000100:00100000:1.0:1789760702.817748:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687019784180 time 687019523764 state 0x1 ref 1 00000100:00100000:1.0:1789760702.817778:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687019784180 time 687138495156 state 0x0 ref 1 00000100:00100000:1.0:1789760702.817792:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495718016:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760702.817839:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20504576, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa007 00010000:00010000:1.0:1789760702.817848:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450ae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.817864:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=687s v=(0 0) 00010000:00010000:1.0:1789760702.817868:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.817875:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.817888:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760702.870618:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20504576 -> 20508672 00010000:00010000:1.0:1789760702.870639:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.870664:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450ae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.870675:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.870682:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760702.870686:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.870695:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450ae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.870714:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450ae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.870723:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450ae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.870821:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495718016:12345-192.168.203.9@tcp:4:dd.0 Request processed in 53025us (53251us total) trans 4294972323 rc 0/0 00000100:00100000:1.0:1789760702.870832:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5035 00000100:00100000:1.0:1789760702.870842:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687019784180 time 687138495156 state 0x2 ref 0 00000100:00100000:2.0:1789760702.874154:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00002000:00010000:2.0:1789760702.939500:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20508672 -> 20512768 00010000:00010000:2.0:1789760702.939521:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545450b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.939534:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545450b5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.939543:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545450b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.939551:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760702.939555:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545450b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.939565:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545450b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.939585:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545450b5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.939594:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545450b5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760702.939914:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495718144:12345-192.168.203.9@tcp:4:dd.0 Request processed in 65507us (65764us total) trans 4294972324 rc 0/0 00000100:00100000:0.0:1789760702.939926:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5036 00000100:00100000:0.0:1789760702.939936:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687138755572 time 687195108048 state 0x2 ref 0 00000100:00100000:0.0:1789760702.951867:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760702.952024:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495718272 00000100:00100000:2.0:1789760702.952034:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495718272 00000100:00100000:2.0:1789760702.952091:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687138755572 time 687195108048 state 0x0 ref 1 00000100:00100000:2.0:1789760702.952107:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687195368464 time 687195108048 state 0x1 ref 1 00000100:00100000:2.0:1789760702.952136:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687195368464 time 687272852712 state 0x0 ref 1 00000100:00100000:2.0:1789760702.952148:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92901237ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495718272:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760702.952197:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20512768, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa007 00010000:00010000:2.0:1789760702.952205:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545450bc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.952222:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=687s v=(0 0) 00010000:00010000:2.0:1789760702.952227:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760702.952233:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545450bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.952247:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545450bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760703.023019:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972325 is committed 00002000:00010000:2.0:1789760703.023214:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20512768 -> 20516864 00010000:00010000:2.0:1789760703.023230:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545450bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.023243:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545450bc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.023253:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545450bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.023267:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760703.023272:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545450bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.023320:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545450bc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.023348:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545450bc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.023356:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545450bc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760703.023436:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92901237ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495718272:12345-192.168.203.9@tcp:4:dd.0 Request processed in 71286us (71574us total) trans 4294972325 rc 0/0 00000100:00100000:2.0:1789760703.023446:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5037 00000100:00100000:2.0:1789760703.023455:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687195368464 time 687272852712 state 0x2 ref 0 00000100:00100000:1.0:1789760703.030624:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495718784 00000100:00100000:1.0:1789760703.030636:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495718784 00000100:00100000:1.0:1789760703.030705:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687195368464 time 687272852712 state 0x0 ref 1 00000100:00100000:1.0:1789760703.030722:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687273113128 time 687272852712 state 0x1 ref 1 00000100:00100000:1.0:1789760703.030754:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687273113128 time 687351470472 state 0x0 ref 1 00000100:00100000:1.0:1789760703.030767:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92902dbc4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495718784:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760703.030815:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20516864, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa007 00010000:00010000:1.0:1789760703.030824:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450c3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.030839:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=687s v=(0 0) 00010000:00010000:1.0:1789760703.030843:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760703.030850:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.030863:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760703.112928:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972326 is committed 00002000:00010000:1.0:1789760703.113060:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20516864 -> 20520960 00010000:00010000:1.0:1789760703.113076:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.113087:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450c3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.113096:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.113102:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760703.113106:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.113114:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450c3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.113140:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450c3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.113148:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450c3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760703.113249:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92902dbc4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495718784:12345-192.168.203.9@tcp:4:dd.0 Request processed in 82480us (83485us total) trans 4294972326 rc 0/0 00000100:00100000:1.0:1789760703.113258:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5038 00000100:00100000:1.0:1789760703.113268:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687273113128 time 687351470472 state 0x2 ref 0 00000100:00100000:0.0:1789760703.123250:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760703.247745:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760703.247857:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495719040 00000100:00100000:2.0:1789760703.247868:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495719040 00000100:00100000:2.0:1789760703.247932:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687351730888 time 687444221547 state 0x0 ref 1 00000100:00100000:2.0:1789760703.247949:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687444481963 time 687444221547 state 0x1 ref 1 00000100:00100000:2.0:1789760703.247985:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687444481963 time 687568702038 state 0x0 ref 1 00000100:00100000:2.0:1789760703.248002:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fdcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495719040:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760703.248051:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20525056, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa007 00010000:00010000:2.0:1789760703.248061:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545450d1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.248077:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=687s v=(0 0) 00010000:00010000:2.0:1789760703.248082:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760703.248087:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545450d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.248101:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545450d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760703.307596:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20525056 -> 20529152 00010000:00010000:2.0:1789760703.307612:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545450d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.307626:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545450d1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.307636:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545450d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.307644:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760703.307650:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545450d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.307659:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545450d1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.307680:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545450d1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.307687:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545450d1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760703.307821:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fdcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495719040:12345-192.168.203.9@tcp:4:dd.0 Request processed in 59816us (60081us total) trans 4294972328 rc 0/0 00000100:00100000:2.0:1789760703.307832:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5040 00000100:00100000:2.0:1789760703.307842:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687444481963 time 687568702038 state 0x2 ref 0 00000100:00100000:1.0:1789760703.313411:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495719168 00000100:00100000:1.0:1789760703.313421:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495719168 00000100:00100000:1.0:1789760703.313475:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687444481963 time 687568702038 state 0x0 ref 1 00000100:00100000:1.0:1789760703.313492:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687568962454 time 687568702038 state 0x1 ref 1 00000100:00100000:1.0:1789760703.313515:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687568962454 time 687634232248 state 0x0 ref 1 00000100:00100000:1.0:1789760703.313527:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903639e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495719168:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760703.313576:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20529152, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa007 00010000:00010000:1.0:1789760703.313585:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450d8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.313601:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=687s v=(0 0) 00010000:00010000:1.0:1789760703.313607:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760703.313614:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.313628:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.326567:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760703.379619:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20529152 -> 20533248 00010000:00010000:2.0:1789760703.379636:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.379655:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450d8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.379665:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.379672:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760703.379679:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.379689:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450d8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.379707:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450d8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.379714:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545450d8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760703.379807:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903639e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495719168:12345-192.168.203.9@tcp:4:dd.0 Request processed in 66277us (66560us total) trans 4294972329 rc 0/0 00000100:00100000:2.0:1789760703.379817:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5041 00000100:00100000:2.0:1789760703.379826:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687568962454 time 687634232248 state 0x2 ref 0 00000100:00100000:2.0:1789760703.383682:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:2.0:1789760703.471075:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972330 is committed 00000100:00100000:2.0:1789760703.475898:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495719424 00000100:00100000:2.0:1789760703.475909:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495719424 00000100:00100000:2.0:1789760703.475970:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687634492664 time 687705637375 state 0x0 ref 1 00000100:00100000:2.0:1789760703.475986:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687705897791 time 687705637375 state 0x1 ref 1 00000100:00100000:2.0:1789760703.476020:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687705897791 time 687796736336 state 0x0 ref 1 00000100:00100000:2.0:1789760703.476032:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92902dbc4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495719424:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760703.476081:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20537344, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa007 00010000:00010000:2.0:1789760703.476090:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545450e6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.476106:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=687s v=(0 0) 00010000:00010000:2.0:1789760703.476111:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760703.476117:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545450e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.476131:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545450e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760703.553959:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20537344 -> 20541440 00010000:00010000:2.0:1789760703.553976:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545450e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.553989:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545450e6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.553999:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545450e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.554006:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760703.554011:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545450e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.554021:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545450e6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.554043:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545450e6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.554052:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545450e6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760703.554137:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92902dbc4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495719424:12345-192.168.203.9@tcp:4:dd.0 Request processed in 78103us (78859us total) trans 4294972331 rc 0/0 00000100:00100000:2.0:1789760703.554146:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5043 00000100:00100000:2.0:1789760703.554157:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687705897791 time 687796736336 state 0x2 ref 0 00000100:00100000:1.0:1789760703.557549:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495719552 00000100:00100000:1.0:1789760703.557559:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495719552 00000100:00100000:1.0:1789760703.557617:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687705897791 time 687796736336 state 0x0 ref 1 00000100:00100000:1.0:1789760703.557632:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687796996752 time 687796736336 state 0x1 ref 1 00000100:00100000:1.0:1789760703.557668:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687796996752 time 687878384735 state 0x0 ref 1 00000100:00100000:1.0:1789760703.557681:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495719552:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760703.557729:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20541440, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa007 00010000:00010000:1.0:1789760703.557738:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545450ed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.557754:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=687s v=(0 0) 00010000:00010000:1.0:1789760703.557759:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760703.557765:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545450ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.557779:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545450ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760703.655430:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972332 is committed 00002000:00010000:0.0:1789760703.657837:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20541440 -> 20545536 00010000:00010000:0.0:1789760703.657859:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545450ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.657872:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545450ed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.657881:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545450ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.657887:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760703.657891:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545450ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.657904:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545450ed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.657924:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545450ed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.657932:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545450ed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760703.658049:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495719552:12345-192.168.203.9@tcp:4:dd.0 Request processed in 100359us (100771us total) trans 4294972332 rc 0/0 00000100:00100000:0.0:1789760703.658060:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5044 00000100:00100000:0.0:1789760703.658070:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687796996752 time 687878384735 state 0x2 ref 0 00000100:00100000:1.0:1789760703.661358:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:0.0:1789760703.749958:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972333 is committed 00002000:00010000:2.0:1789760703.750187:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20545536 -> 20549632 00002000:00010000:2.0:1789760703.750192:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760702 -> 1789760703 00002000:00010000:2.0:1789760703.750194:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760702 -> 1789760703 00010000:00010000:2.0:1789760703.750208:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545450f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.750221:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545450f4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.750230:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545450f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.750237:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760703.750243:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545450f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.750252:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545450f4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.750275:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545450f4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.750319:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545450f4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760703.750420:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495719680:12345-192.168.203.9@tcp:4:dd.0 Request processed in 88850us (89067us total) trans 4294972333 rc 0/0 00000100:00100000:2.0:1789760703.750431:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5045 00000100:00100000:2.0:1789760703.750441:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687878645151 time 687982271177 state 0x2 ref 0 00000100:00100000:2.0:1789760703.753854:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495719808 00000100:00100000:2.0:1789760703.753865:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495719808 00000100:00100000:2.0:1789760703.753923:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687878645151 time 687982271177 state 0x0 ref 1 00000100:00100000:2.0:1789760703.753940:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687982531593 time 687982271177 state 0x1 ref 1 00000100:00100000:2.0:1789760703.753970:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687982531593 time 688074686666 state 0x0 ref 1 00000100:00100000:2.0:1789760703.753982:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495719808:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760703.754033:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20549632, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa007 00010000:00010000:2.0:1789760703.754042:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545450fb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.754062:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:2.0:1789760703.754067:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760703.754073:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545450fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.754087:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545450fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760703.865829:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20549632 -> 20553728 00010000:00010000:2.0:1789760703.865848:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545450fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.865862:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545450fb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.865872:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545450fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.865879:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760703.865883:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545450fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.865892:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545450fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.865910:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545450fb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.865917:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545450fb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760703.866012:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495719808:12345-192.168.203.9@tcp:4:dd.0 Request processed in 112026us (112501us total) trans 4294972334 rc 0/0 00000100:00100000:2.0:1789760703.866021:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5046 00000100:00100000:2.0:1789760703.866031:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687982531593 time 688074686666 state 0x2 ref 0 00000001:00080000:0.0:1789760703.869449:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972334 is committed 00000100:00100000:0.0:1789760703.870833:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495719936 00000100:00100000:0.0:1789760703.870844:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495719936 00000100:00100000:0.0:1789760703.870903:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 687982531593 time 688074686666 state 0x0 ref 1 00000100:00100000:0.0:1789760703.870918:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688074947082 time 688074686666 state 0x1 ref 1 00000100:00100000:0.0:1789760703.870955:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688074947082 time 688191672327 state 0x0 ref 1 00000100:00100000:0.0:1789760703.870968:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495719936:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760703.871019:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20553728, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa007 00010000:00010000:0.0:1789760703.871028:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545102 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.871043:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:0.0:1789760703.871046:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760703.871053:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545102 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.871066:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545102 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760703.977217:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972335 is committed 00002000:00010000:2.0:1789760703.977400:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20553728 -> 20557824 00010000:00010000:2.0:1789760703.977489:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545102 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.977501:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545102 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.977511:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545102 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.977518:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760703.977523:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545102 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.977532:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545102 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.977557:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545102 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.977565:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545102 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760703.978068:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495719936:12345-192.168.203.9@tcp:4:dd.0 Request processed in 107095us (107986us total) trans 4294972335 rc 0/0 00000100:00100000:2.0:1789760703.978079:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5047 00000100:00100000:2.0:1789760703.978090:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688074947082 time 688191672327 state 0x2 ref 0 00000100:00100000:2.0:1789760703.980875:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495720064 00000100:00100000:2.0:1789760703.980885:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495720064 00000100:00100000:2.0:1789760703.980940:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688074947082 time 688191672327 state 0x0 ref 1 00000100:00100000:2.0:1789760703.980958:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688191932743 time 688191672327 state 0x1 ref 1 00000100:00100000:2.0:1789760703.980989:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688191932743 time 688301705501 state 0x0 ref 1 00000100:00100000:2.0:1789760703.981001:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495720064:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760703.981052:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20557824, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa007 00010000:00010000:2.0:1789760703.981062:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545109 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.981079:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:2.0:1789760703.981084:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760703.981091:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545109 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.981109:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545109 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760704.061455:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20557824 -> 20561920 00010000:00010000:2.0:1789760704.061471:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545109 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.061484:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545109 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.061493:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545109 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.061501:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760704.061506:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545109 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.061515:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545109 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.061536:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545109 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.061549:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545109 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.061655:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495720064:12345-192.168.203.9@tcp:4:dd.0 Request processed in 80651us (80879us total) trans 4294972336 rc 0/0 00000100:00100000:2.0:1789760704.061666:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5048 00000100:00100000:2.0:1789760704.061675:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688191932743 time 688301705501 state 0x2 ref 0 00000100:00100000:2.0:1789760704.072710:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495720192 00000100:00100000:2.0:1789760704.072720:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495720192 00000100:00100000:2.0:1789760704.072774:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688191932743 time 688301705501 state 0x0 ref 1 00000100:00100000:2.0:1789760704.072791:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688301965917 time 688301705501 state 0x1 ref 1 00000100:00100000:2.0:1789760704.072876:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688301965917 time 688393592491 state 0x0 ref 1 00000100:00100000:2.0:1789760704.072889:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495720192:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760704.072952:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20561920, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa007 00010000:00010000:2.0:1789760704.072962:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545110 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.072978:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:2.0:1789760704.072983:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.072989:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545110 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.073011:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545110 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760704.108200:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972337 is committed 00002000:00010000:2.0:1789760704.108482:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20561920 -> 20566016 00010000:00010000:2.0:1789760704.108499:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545110 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.108513:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545110 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.108521:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545110 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.108528:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760704.108533:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545110 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.108542:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545110 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.108562:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545110 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.108571:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545110 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.108665:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495720192:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35774us (36428us total) trans 4294972337 rc 0/0 00000100:00100000:2.0:1789760704.108676:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5049 00000100:00100000:2.0:1789760704.108685:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688301965917 time 688393592491 state 0x2 ref 0 00000100:00100000:2.0:1789760704.118092:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495720320 00000100:00100000:2.0:1789760704.118101:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495720320 00000100:00100000:2.0:1789760704.118156:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688301965917 time 688393592491 state 0x0 ref 1 00000100:00100000:2.0:1789760704.118172:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688393852907 time 688393592491 state 0x1 ref 1 00000100:00100000:2.0:1789760704.118206:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688393852907 time 688438922943 state 0x0 ref 1 00000100:00100000:2.0:1789760704.118219:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495720320:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760704.118268:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20566016, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa007 00010000:00010000:2.0:1789760704.118276:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545117 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.118454:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:2.0:1789760704.118459:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.118467:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545117 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.118483:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545117 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760704.203445:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20566016 -> 20570112 00010000:00010000:0.0:1789760704.203461:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545117 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.203475:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545117 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.203485:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545117 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.203493:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760704.203497:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545117 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.203513:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545117 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.203532:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545117 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.203539:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545117 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760704.203628:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495720320:12345-192.168.203.9@tcp:4:dd.0 Request processed in 85406us (85908us total) trans 4294972338 rc 0/0 00000100:00100000:0.0:1789760704.203639:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5050 00000100:00100000:0.0:1789760704.205502:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688393852907 time 688438922943 state 0x2 ref 0 00000100:00100000:2.0:1789760704.208776:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495720448 00000100:00100000:2.0:1789760704.208786:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495720448 00000100:00100000:2.0:1789760704.208844:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688393852907 time 688438922943 state 0x0 ref 1 00000100:00100000:2.0:1789760704.208860:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688439183359 time 688438922943 state 0x1 ref 1 00000100:00100000:2.0:1789760704.208888:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688439183359 time 688529605534 state 0x0 ref 1 00000100:00100000:2.0:1789760704.208901:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495720448:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760704.208947:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20570112, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa007 00010000:00010000:2.0:1789760704.208956:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454511e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.208972:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:2.0:1789760704.208976:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.208982:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454511e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.208995:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454511e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760704.300477:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972339 is committed 00002000:00010000:1.0:1789760704.303864:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20570112 -> 20574208 00010000:00010000:1.0:1789760704.303885:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454511e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.303898:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454511e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.303908:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454511e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.303915:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760704.303920:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454511e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.303929:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454511e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.303949:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454511e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.303956:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454511e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760704.304123:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495720448:12345-192.168.203.9@tcp:4:dd.0 Request processed in 95218us (95548us total) trans 4294972339 rc 0/0 00000100:00100000:1.0:1789760704.304134:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5051 00000100:00100000:1.0:1789760704.304144:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688439183359 time 688529605534 state 0x2 ref 0 00000100:00100000:0.0:1789760704.307555:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495720576 00000100:00100000:0.0:1789760704.307565:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495720576 00000100:00100000:0.0:1789760704.307619:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688439183359 time 688529605534 state 0x0 ref 1 00000100:00100000:0.0:1789760704.307635:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688529865950 time 688529605534 state 0x1 ref 1 00000100:00100000:0.0:1789760704.307734:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688529865950 time 688628450076 state 0x0 ref 1 00000100:00100000:0.0:1789760704.307749:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495720576:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760704.307797:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20574208, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa007 00010000:00010000:0.0:1789760704.307807:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545125 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.307823:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:0.0:1789760704.307828:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760704.307834:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545125 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.307849:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545125 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.369172:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760704.374758:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972340 is committed 00002000:00010000:0.0:1789760704.375163:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20574208 -> 20578304 00010000:00010000:0.0:1789760704.375181:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545125 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.375196:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545125 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.375205:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545125 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.375213:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760704.375219:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545125 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.375229:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545125 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.375252:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545125 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.375261:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545125 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760704.375432:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495720576:12345-192.168.203.9@tcp:4:dd.0 Request processed in 67680us (68025us total) trans 4294972340 rc 0/0 00000100:00100000:0.0:1789760704.375442:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5052 00000100:00100000:0.0:1789760704.375451:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688529865950 time 688628450076 state 0x2 ref 0 00000100:00100000:0.0:1789760704.380920:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495720704 00000100:00100000:0.0:1789760704.380930:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495720704 00000100:00100000:0.0:1789760704.380991:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688529865950 time 688628450076 state 0x0 ref 1 00000100:00100000:0.0:1789760704.381005:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688628710492 time 688628450076 state 0x1 ref 1 00000100:00100000:0.0:1789760704.381041:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688628710492 time 688701757837 state 0x0 ref 1 00000100:00100000:0.0:1789760704.381053:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495720704:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760704.381102:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20578304, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa007 00010000:00010000:0.0:1789760704.381111:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454512c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.381126:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:0.0:1789760704.381131:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760704.381137:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454512c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.381157:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454512c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760704.464962:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972341 is committed 00002000:00010000:0.0:1789760704.464963:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20578304 -> 20582400 00010000:00010000:0.0:1789760704.464984:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454512c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.464998:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454512c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.465007:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454512c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.465015:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760704.465020:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454512c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.465029:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454512c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.465048:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454512c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.465056:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454512c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760704.465156:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495720704:12345-192.168.203.9@tcp:4:dd.0 Request processed in 84100us (84344us total) trans 4294972341 rc 0/0 00000100:00100000:0.0:1789760704.465167:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5053 00000100:00100000:0.0:1789760704.465177:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688628710492 time 688701757837 state 0x2 ref 0 00000100:00100000:1.0:1789760704.469947:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495720832 00000100:00100000:1.0:1789760704.469958:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495720832 00000100:00100000:1.0:1789760704.470012:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688628710492 time 688701757837 state 0x0 ref 1 00000100:00100000:1.0:1789760704.470027:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688702018253 time 688701757837 state 0x1 ref 1 00000100:00100000:1.0:1789760704.470057:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688702018253 time 688790773822 state 0x0 ref 1 00000100:00100000:1.0:1789760704.470070:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495720832:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760704.470112:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20582400, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa007 00010000:00010000:1.0:1789760704.470121:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545133 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.470137:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:1.0:1789760704.470141:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760704.470148:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545133 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.470161:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545133 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760704.523740:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972342 is committed 00000100:00100000:0.0:1789760704.526628:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495720960 00000100:00100000:0.0:1789760704.526649:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495720960 00000100:00100000:0.0:1789760704.526700:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688702018253 time 688790773822 state 0x0 ref 1 00000100:00100000:0.0:1789760704.526716:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688791034238 time 688790773822 state 0x1 ref 1 00000100:00100000:0.0:1789760704.526745:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688791034238 time 688847462374 state 0x0 ref 1 00000100:00100000:0.0:1789760704.526755:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f921c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495720960:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760704.526810:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20586496, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa007 00010000:00010000:0.0:1789760704.526820:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454513a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.526839:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:0.0:1789760704.526843:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760704.526850:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454513a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.526866:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454513a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760704.554374:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972343 is committed 00002000:00010000:1.0:1789760704.554732:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20586496 -> 20590592 00010000:00010000:1.0:1789760704.554749:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454513a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.554761:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454513a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.554770:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454513a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.554777:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760704.554781:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454513a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.554790:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454513a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.554809:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454513a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.554817:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454513a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760704.554905:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f921c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495720960:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28145us (28682us total) trans 4294972343 rc 0/0 00000100:00100000:1.0:1789760704.554919:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5055 00000100:00100000:1.0:1789760704.554928:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688791034238 time 688847462374 state 0x2 ref 0 00000100:00100000:2.0:1789760704.558758:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760704.559253:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495721088 00000100:00100000:0.0:1789760704.559262:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495721088 00000100:00100000:0.0:1789760704.559610:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688791034238 time 688847462374 state 0x0 ref 1 00000100:00100000:0.0:1789760704.559625:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688847722790 time 688847462374 state 0x1 ref 1 00000100:00100000:0.0:1789760704.559665:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688847722790 time 688880381854 state 0x0 ref 1 00000100:00100000:0.0:1789760704.559679:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495721088:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760704.559725:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20590592, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa007 00010000:00010000:0.0:1789760704.559734:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545141 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.559751:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:0.0:1789760704.559755:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760704.559761:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545141 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.559775:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545141 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760704.626241:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20590592 -> 20594688 00010000:00010000:2.0:1789760704.626263:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545141 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.626276:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545141 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.626336:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545141 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.626344:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760704.626350:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545141 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.626358:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545141 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.626387:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545141 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.626396:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545141 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.626486:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495721088:12345-192.168.203.9@tcp:4:dd.0 Request processed in 66804us (67731us total) trans 4294972344 rc 0/0 00000100:00100000:2.0:1789760704.626495:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5056 00000100:00100000:2.0:1789760704.626505:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688847722790 time 688880381854 state 0x2 ref 0 00000001:00080000:2.0:1789760704.626778:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972344 is committed 00000100:00100000:1.0:1789760704.635924:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495721216 00000100:00100000:1.0:1789760704.635933:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495721216 00000100:00100000:1.0:1789760704.636048:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688847722790 time 688880381854 state 0x0 ref 1 00000100:00100000:1.0:1789760704.636064:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688880642270 time 688880381854 state 0x1 ref 1 00000100:00100000:1.0:1789760704.636096:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688880642270 time 688956812973 state 0x0 ref 1 00000100:00100000:1.0:1789760704.636107:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f923100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495721216:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760704.636153:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20594688, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa007 00010000:00010000:1.0:1789760704.636161:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545148 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.636177:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:1.0:1789760704.636181:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760704.636188:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545148 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.636202:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545148 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760704.669164:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972345 is committed 00000100:00100000:1.0:1789760704.673199:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760704.673390:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495721344 00000100:00100000:0.0:1789760704.673399:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495721344 00000100:00100000:0.0:1789760704.673454:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688880642270 time 688956812973 state 0x0 ref 1 00000100:00100000:0.0:1789760704.673469:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688957073389 time 688956812973 state 0x1 ref 1 00000100:00100000:0.0:1789760704.673578:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688957073389 time 688994294665 state 0x0 ref 1 00000100:00100000:0.0:1789760704.673591:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495721344:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760704.673640:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20598784, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa007 00010000:00010000:0.0:1789760704.673658:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454514f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.673675:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:0.0:1789760704.673679:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760704.673686:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454514f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.673701:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454514f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760704.723758:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972346 is committed 00002000:00010000:1.0:1789760704.723839:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20598784 -> 20602880 00010000:00010000:1.0:1789760704.723851:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454514f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.723861:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454514f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.723866:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454514f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.723871:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760704.723874:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454514f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.723880:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454514f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.723896:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454514f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.723900:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454514f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760704.724041:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495721344:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50449us (50849us total) trans 4294972346 rc 0/0 00000100:00100000:1.0:1789760704.724050:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5058 00000100:00100000:1.0:1789760704.724059:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688957073389 time 688994294665 state 0x2 ref 0 00000100:00100000:2.0:1789760704.729857:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760704.730075:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495721472 00000100:00100000:0.0:1789760704.730084:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495721472 00000100:00100000:0.0:1789760704.730141:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688957073389 time 688994294665 state 0x0 ref 1 00000100:00100000:0.0:1789760704.730156:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688994555081 time 688994294665 state 0x1 ref 1 00000100:00100000:0.0:1789760704.730226:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688994555081 time 689050941833 state 0x0 ref 1 00000100:00100000:0.0:1789760704.730460:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495721472:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760704.730507:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20602880, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa007 00010000:00010000:0.0:1789760704.730516:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545156 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.730532:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:0.0:1789760704.730537:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760704.730543:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545156 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.730557:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545156 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760704.779102:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20602880 -> 20606976 00010000:00010000:2.0:1789760704.779122:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545156 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.779135:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545156 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.779144:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545156 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.779151:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760704.779156:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545156 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.779164:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545156 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.779181:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545156 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.779188:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545156 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.779442:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495721472:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48915us (49523us total) trans 4294972347 rc 0/0 00000100:00100000:2.0:1789760704.779454:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5059 00000100:00100000:2.0:1789760704.779464:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688994555081 time 689050941833 state 0x2 ref 0 00000100:00100000:1.0:1789760704.789633:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495721600 00000100:00100000:1.0:1789760704.789643:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495721600 00000100:00100000:1.0:1789760704.789869:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 688994555081 time 689050941833 state 0x0 ref 1 00000100:00100000:1.0:1789760704.789927:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689051202249 time 689050941833 state 0x1 ref 1 00000100:00100000:1.0:1789760704.789960:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689051202249 time 689110676196 state 0x0 ref 1 00000100:00100000:1.0:1789760704.789974:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f922300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495721600:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760704.790031:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20606976, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa007 00010000:00010000:1.0:1789760704.790041:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454515d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.790212:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:1.0:1789760704.790217:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760704.790225:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454515d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.790241:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454515d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760704.798409:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760704.798432:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520299776:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760704.798502:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760704.798580:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520299776 00000100:00100000:1.0:1789760704.798587:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520299776 00000100:00100000:1.0:1789760704.798655:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 185 00000100:00100000:1.0:1789760704.798665:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520299776:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760704.798688:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527761408 pending 1703936 free 3837788160 avail 3834642432 00000100:00100000:1.0:1789760704.798734:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520299776:12345-0@lo:13:osp-pre-0-0.0 Request processed in 71us (233us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760704.798741:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 185 00000100:00080000:1.0:1789760704.798765:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760704.798934:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520299776:0@lo:13:osp-pre-0-0.0 00002000:00010000:2.0:1789760704.835699:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20606976 -> 20611072 00010000:00010000:2.0:1789760704.835717:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454515d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.835726:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454515d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.835733:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454515d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.835737:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760704.835741:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454515d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.835747:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454515d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.835761:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454515d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.835776:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454515d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.835856:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f922300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495721600:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45881us (46976us total) trans 4294972348 rc 0/0 00000100:00100000:2.0:1789760704.835863:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5060 00000100:00100000:2.0:1789760704.835871:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689051202249 time 689110676196 state 0x2 ref 0 00000100:00100000:0.0:1789760704.840718:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760704.840890:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495721728 00000100:00100000:1.0:1789760704.840901:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495721728 00000100:00100000:1.0:1789760704.840956:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689051202249 time 689110676196 state 0x0 ref 1 00000100:00100000:1.0:1789760704.840972:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689110936612 time 689110676196 state 0x1 ref 1 00000100:00100000:1.0:1789760704.841028:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689110936612 time 689161745639 state 0x0 ref 1 00000100:00100000:1.0:1789760704.841037:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fdc9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495721728:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760704.841088:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20611072, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa007 00010000:00010000:1.0:1789760704.841098:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545164 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.841115:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:1.0:1789760704.841120:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760704.841127:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545164 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.841147:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545164 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760704.875978:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20611072 -> 20615168 00010000:00010000:0.0:1789760704.875997:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545164 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.876011:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545164 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.876021:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545164 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.876029:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760704.876034:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545164 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.876044:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545164 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.876064:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545164 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.876072:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545164 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760704.876164:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fdc9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495721728:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35122us (35451us total) trans 4294972349 rc 0/0 00000100:00100000:0.0:1789760704.876175:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5061 00000100:00100000:0.0:1789760704.876184:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689110936612 time 689161745639 state 0x2 ref 0 00000001:00080000:2.0:1789760704.877596:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972349 is committed 00000100:00100000:0.0:1789760704.881584:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760704.881686:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495721856 00000100:00100000:2.0:1789760704.881695:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495721856 00000100:00100000:2.0:1789760704.881747:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689110936612 time 689161745639 state 0x0 ref 1 00000100:00100000:2.0:1789760704.881762:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689162006055 time 689161745639 state 0x1 ref 1 00000100:00100000:2.0:1789760704.881791:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689162006055 time 689202508493 state 0x0 ref 1 00000100:00100000:2.0:1789760704.881803:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fdcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495721856:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760704.881853:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20615168, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa007 00010000:00010000:2.0:1789760704.881863:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454516b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.881880:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:2.0:1789760704.881885:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.881892:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454516b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.881908:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454516b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760704.920449:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972350 is committed 00002000:00010000:2.0:1789760704.920452:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20615168 -> 20619264 00010000:00010000:2.0:1789760704.920467:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454516b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.920481:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454516b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.920491:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454516b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.920499:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760704.920504:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454516b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.920514:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454516b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.920534:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454516b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.920542:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454516b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.920638:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fdcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495721856:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38831us (39058us total) trans 4294972350 rc 0/0 00000100:00100000:2.0:1789760704.920650:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5062 00000100:00100000:2.0:1789760704.920661:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689162006055 time 689202508493 state 0x2 ref 0 00000100:00100000:1.0:1789760704.924229:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760704.924479:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495721984 00000100:00100000:2.0:1789760704.924490:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495721984 00000100:00100000:2.0:1789760704.924549:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689162006055 time 689202508493 state 0x0 ref 1 00000100:00100000:2.0:1789760704.924566:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689202768909 time 689202508493 state 0x1 ref 1 00000100:00100000:2.0:1789760704.924588:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689202768909 time 689245305061 state 0x0 ref 1 00000100:00100000:2.0:1789760704.924601:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495721984:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760704.924746:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20619264, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa007 00010000:00010000:2.0:1789760704.925170:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545172 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.925189:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:2.0:1789760704.925194:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.925203:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545172 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.925217:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545172 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760704.958723:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972351 is committed 00002000:00010000:2.0:1789760704.958959:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20619264 -> 20623360 00010000:00010000:2.0:1789760704.958977:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545172 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.958992:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545172 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.959002:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545172 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.959009:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760704.959015:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545172 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.959024:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545172 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.959046:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545172 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.959054:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545172 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.959164:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495721984:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34560us (34940us total) trans 4294972351 rc 0/0 00000100:00100000:2.0:1789760704.959179:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5063 00000100:00100000:2.0:1789760704.959189:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689202768909 time 689245305061 state 0x2 ref 0 00000100:00100000:0.0:1789760704.970617:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760704.970717:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495722112 00000100:00100000:1.0:1789760704.970727:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495722112 00000100:00100000:1.0:1789760704.970779:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689202768909 time 689245305061 state 0x0 ref 1 00000100:00100000:1.0:1789760704.970795:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689245565477 time 689245305061 state 0x1 ref 1 00000100:00100000:1.0:1789760704.970825:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689245565477 time 689291542291 state 0x0 ref 1 00000100:00100000:1.0:1789760704.970838:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495722112:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760704.970887:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20623360, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa007 00010000:00010000:1.0:1789760704.970896:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545179 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.970912:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:1.0:1789760704.970917:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760704.970924:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545179 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.970937:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545179 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760704.996646:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972352 is committed 00002000:00010000:2.0:1789760704.996818:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20623360 -> 20627456 00010000:00010000:2.0:1789760704.996833:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545179 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.996844:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545179 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.996852:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545179 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.996859:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760704.996865:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545179 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.996874:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545179 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.996894:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545179 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.996906:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545179 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.997031:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495722112:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26193us (26420us total) trans 4294972352 rc 0/0 00000100:00100000:2.0:1789760704.997039:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5064 00000100:00100000:2.0:1789760704.997047:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689245565477 time 689291542291 state 0x2 ref 0 00000100:00100000:0.0:1789760705.003771:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760705.003925:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495722240 00000100:00100000:2.0:1789760705.003934:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495722240 00000100:00100000:2.0:1789760705.004006:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689245565477 time 689291542291 state 0x0 ref 1 00000100:00100000:2.0:1789760705.004022:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689291802707 time 689291542291 state 0x1 ref 1 00000100:00080000:1.0:1789760705.004043:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760705.004050:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689291802707 time 689324765652 state 0x0 ref 1 00000100:00100000:2.0:1789760705.004065:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495722240:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760705.004110:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20627456, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa007 00010000:00010000:2.0:1789760705.004118:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545180 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.004133:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:2.0:1789760705.004137:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.004142:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545180 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.004155:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545180 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760705.031256:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20627456 -> 20631552 00010000:00010000:2.0:1789760705.031270:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545180 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.031366:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545180 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.031375:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545180 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.031382:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760705.031386:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545180 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.031396:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545180 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.031416:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545180 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.031424:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545180 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.031511:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495722240:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27446us (27745us total) trans 4294972353 rc 0/0 00000100:00100000:2.0:1789760705.031527:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5065 00000100:00100000:2.0:1789760705.031539:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689291802707 time 689324765652 state 0x2 ref 0 00000100:00100000:2.0:1789760705.036478:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495722368 00000100:00100000:2.0:1789760705.036488:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495722368 00000100:00100000:2.0:1789760705.036547:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689291802707 time 689324765652 state 0x0 ref 1 00000100:00100000:2.0:1789760705.036561:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689325026068 time 689324765652 state 0x1 ref 1 00000100:00100000:2.0:1789760705.036588:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689325026068 time 689357304541 state 0x0 ref 1 00000100:00100000:2.0:1789760705.036599:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495722368:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760705.036647:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20631552, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa007 00010000:00010000:2.0:1789760705.036655:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545187 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.036671:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:2.0:1789760705.036676:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.036681:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545187 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.036694:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545187 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760705.053756:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972354 is committed 00000100:00100000:0.0:1789760705.062046:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760705.062126:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495722496 00000100:00100000:2.0:1789760705.062136:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495722496 00000100:00100000:2.0:1789760705.062191:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689325026068 time 689357304541 state 0x0 ref 1 00000100:00100000:2.0:1789760705.062207:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689357564957 time 689357304541 state 0x1 ref 1 00000100:00100000:2.0:1789760705.062232:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689357564957 time 689382949321 state 0x0 ref 1 00000100:00100000:2.0:1789760705.062245:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01c8c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495722496:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760705.062598:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20635648, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa007 00010000:00010000:2.0:1789760705.062608:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454518e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.062625:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:2.0:1789760705.062630:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.062637:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454518e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.062651:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454518e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760705.088168:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20635648 -> 20639744 00010000:00010000:2.0:1789760705.088187:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454518e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.088200:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454518e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.088219:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454518e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.088229:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760705.088234:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454518e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.088704:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454518e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.088727:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454518e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.088737:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454518e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.088933:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01c8c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495722496:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26686us (26892us total) trans 4294972355 rc 0/0 00000100:00100000:2.0:1789760705.088944:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5067 00000100:00100000:2.0:1789760705.088954:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689357564957 time 689382949321 state 0x2 ref 0 00000100:00100000:0.0:1789760705.093575:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760705.093686:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495722624 00000100:00100000:2.0:1789760705.093697:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495722624 00000100:00100000:2.0:1789760705.093757:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689357564957 time 689382949321 state 0x0 ref 1 00000100:00100000:2.0:1789760705.093773:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689383209737 time 689382949321 state 0x1 ref 1 00000100:00100000:2.0:1789760705.093808:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689383209737 time 689414524437 state 0x0 ref 1 00000100:00100000:2.0:1789760705.093823:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01c8d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495722624:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760705.093871:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20639744, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa007 00010000:00010000:2.0:1789760705.093879:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545195 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.093896:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:2.0:1789760705.093901:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.093908:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545195 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.093922:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545195 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760705.145837:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760705.189504:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760705.191968:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495722880 00000100:00100000:2.0:1789760705.191982:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495722880 00000100:00100000:2.0:1789760705.192046:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689414784853 time 689466745693 state 0x0 ref 1 00000100:00100000:2.0:1789760705.192061:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689467006109 time 689466745693 state 0x1 ref 1 00000100:00100000:2.0:1789760705.192094:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689467006109 time 689512811697 state 0x0 ref 1 00000100:00100000:2.0:1789760705.192108:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495722880:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760705.192155:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20647936, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa007 00010000:00010000:2.0:1789760705.192164:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545451a3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.192185:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:2.0:1789760705.192189:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.192196:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545451a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.192451:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545451a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760705.232597:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20647936 -> 20652032 00010000:00010000:2.0:1789760705.232617:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545451a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.232628:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545451a3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.232636:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545451a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.232642:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760705.232647:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545451a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.232655:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545451a3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.232675:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545451a3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.232683:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545451a3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.232773:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495722880:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40662us (43274us total) trans 4294972358 rc 0/0 00000100:00100000:2.0:1789760705.232783:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5070 00000100:00100000:2.0:1789760705.232793:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689467006109 time 689512811697 state 0x2 ref 0 00000100:00100000:1.0:1789760705.242395:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760705.280910:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:2.0:1789760705.320496:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972360 is committed 00000100:00100000:1.0:1789760705.328928:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760705.329189:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495723264 00000100:00100000:2.0:1789760705.329200:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495723264 00000100:00100000:2.0:1789760705.329253:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689563872765 time 689601897258 state 0x0 ref 1 00000100:00100000:2.0:1789760705.329274:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689602157674 time 689601897258 state 0x1 ref 1 00000100:00100000:2.0:1789760705.329353:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689602157674 time 689650065604 state 0x0 ref 1 00000100:00100000:2.0:1789760705.329376:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495723264:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760705.329425:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20660224, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa007 00010000:00010000:2.0:1789760705.329435:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545451b8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.329452:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:2.0:1789760705.329456:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.329463:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545451b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.329477:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545451b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760705.354475:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972361 is committed 00010000:00010000:3.0:1789760705.354704:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545451b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.354718:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760705.354723:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545451b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.354732:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545451b8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.354753:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545451b8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.354762:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545451b8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760705.354883:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495723264:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25505us (25963us total) trans 4294972361 rc 0/0 00000100:00100000:3.0:1789760705.354894:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5073 00000100:00100000:3.0:1789760705.354905:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689602157674 time 689650065604 state 0x2 ref 0 00000100:00100000:1.0:1789760705.366895:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760705.366971:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495723392 00000100:00100000:3.0:1789760705.366983:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495723392 00000100:00100000:3.0:1789760705.367036:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689602157674 time 689650065604 state 0x0 ref 1 00000100:00100000:3.0:1789760705.367050:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689650326020 time 689650065604 state 0x1 ref 1 00000100:00100000:3.0:1789760705.367089:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689650326020 time 689687804951 state 0x0 ref 1 00000100:00100000:3.0:1789760705.367107:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495723392:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760705.367159:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20664320, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa007 00010000:00010000:3.0:1789760705.367169:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545451bf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.367186:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:3.0:1789760705.367191:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760705.367198:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545451bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.367214:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545451bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.374419:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760705.399766:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972362 is committed 00002000:00010000:3.0:1789760705.399799:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20664320 -> 20668416 00010000:00010000:3.0:1789760705.399816:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545451bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.399829:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545451bf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.399837:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545451bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.399845:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760705.399850:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545451bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.399861:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545451bf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.399881:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545451bf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.399889:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545451bf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760705.400017:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495723392:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32910us (33245us total) trans 4294972362 rc 0/0 00000100:00100000:3.0:1789760705.400028:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5074 00000100:00100000:3.0:1789760705.400037:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689650326020 time 689687804951 state 0x2 ref 0 00000100:00100000:2.0:1789760705.410473:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760705.410594:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495723520 00000100:00100000:3.0:1789760705.410604:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495723520 00000100:00100000:3.0:1789760705.410659:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689650326020 time 689687804951 state 0x0 ref 1 00000100:00100000:3.0:1789760705.410676:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689688065367 time 689687804951 state 0x1 ref 1 00000100:00100000:0.0:1789760705.410812:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689688065367 time 689731525636 state 0x0 ref 1 00000100:00100000:0.0:1789760705.410827:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495723520:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760705.410872:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20668416, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa007 00010000:00010000:0.0:1789760705.410881:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545451c6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.410898:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:0.0:1789760705.410902:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760705.410908:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545451c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.410922:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545451c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760705.437079:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972363 is committed 00002000:00010000:2.0:1789760705.437143:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20668416 -> 20672512 00010000:00010000:2.0:1789760705.437156:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545451c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.437166:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545451c6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.437171:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545451c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.437176:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760705.437180:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545451c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.437186:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545451c6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.437202:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545451c6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.437210:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545451c6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.437342:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495723520:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26514us (26873us total) trans 4294972363 rc 0/0 00000100:00100000:2.0:1789760705.437353:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5075 00000100:00100000:2.0:1789760705.437362:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689688065367 time 689731525636 state 0x2 ref 0 00000100:00100000:1.0:1789760705.442435:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760705.443012:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495723648 00000100:00100000:3.0:1789760705.443034:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495723648 00000100:00100000:3.0:1789760705.443654:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689688065367 time 689731525636 state 0x0 ref 1 00000100:00100000:3.0:1789760705.443675:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689731786052 time 689731525636 state 0x1 ref 1 00000100:00100000:3.0:1789760705.443709:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689731786052 time 689764426105 state 0x0 ref 1 00000100:00100000:3.0:1789760705.443722:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495723648:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760705.443962:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20672512, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa007 00010000:00010000:3.0:1789760705.443972:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545451cd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.443988:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:3.0:1789760705.443992:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760705.443998:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545451cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.444011:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545451cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760705.520536:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972364 is committed 00002000:00010000:2.0:1789760705.520672:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20672512 -> 20676608 00010000:00010000:2.0:1789760705.520687:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545451cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.520699:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545451cd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.520708:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545451cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.520716:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760705.520720:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545451cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.520730:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545451cd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.520754:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545451cd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.520762:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545451cd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.520868:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495723648:12345-192.168.203.9@tcp:4:dd.0 Request processed in 77143us (78435us total) trans 4294972364 rc 0/0 00000100:00100000:2.0:1789760705.520877:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5076 00000100:00100000:2.0:1789760705.520887:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689731786052 time 689764426105 state 0x2 ref 0 00000100:00100000:1.0:1789760705.542371:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760705.542892:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495723776 00000100:00100000:3.0:1789760705.542903:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495723776 00000100:00100000:3.0:1789760705.542960:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689731786052 time 689764426105 state 0x0 ref 1 00000100:00100000:3.0:1789760705.542975:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689764686521 time 689764426105 state 0x1 ref 1 00000100:00100000:3.0:1789760705.543011:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689764686521 time 689863727934 state 0x0 ref 1 00000100:00100000:3.0:1789760705.543024:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495723776:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760705.543086:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20676608, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa007 00010000:00010000:3.0:1789760705.543097:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545451d4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.543112:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:3.0:1789760705.543116:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760705.543122:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545451d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.543136:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545451d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760705.613658:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972365 is committed 00002000:00010000:2.0:1789760705.613811:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20676608 -> 20680704 00010000:00010000:2.0:1789760705.613828:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545451d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.613841:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545451d4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.613849:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545451d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.613855:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760705.613860:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545451d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.613874:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545451d4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.613892:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545451d4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.613899:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545451d4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.614022:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495723776:12345-192.168.203.9@tcp:4:dd.0 Request processed in 70995us (71654us total) trans 4294972365 rc 0/0 00000100:00100000:2.0:1789760705.614033:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5077 00000100:00100000:2.0:1789760705.614115:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689764686521 time 689863727934 state 0x2 ref 0 00000100:00100000:0.0:1789760705.617953:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760705.618108:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495723904 00000100:00100000:2.0:1789760705.618118:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495723904 00000100:00100000:2.0:1789760705.618174:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689764686521 time 689863727934 state 0x0 ref 1 00000100:00100000:2.0:1789760705.618189:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689863988350 time 689863727934 state 0x1 ref 1 00000100:00100000:2.0:1789760705.618260:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689863988350 time 689938975830 state 0x0 ref 1 00000100:00100000:2.0:1789760705.618273:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495723904:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760705.618411:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20680704, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa007 00010000:00010000:2.0:1789760705.618420:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545451db lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.618436:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:2.0:1789760705.618440:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.618446:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545451db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.618459:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545451db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760705.649036:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20680704 -> 20684800 00002000:00010000:0.0:1789760705.649046:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760704 -> 1789760705 00002000:00010000:0.0:1789760705.649049:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760704 -> 1789760705 00010000:00010000:0.0:1789760705.649062:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545451db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.649076:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545451db lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.649085:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545451db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.649093:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760705.649098:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545451db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.649113:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545451db lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.649133:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545451db lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.649140:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545451db lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760705.649363:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495723904:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31086us (31412us total) trans 4294972366 rc 0/0 00000100:00100000:0.0:1789760705.649374:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5078 00000100:00100000:0.0:1789760705.649384:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689863988350 time 689938975830 state 0x2 ref 0 00000001:00080000:0.0:1789760705.649443:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972366 is committed 00000100:00100000:1.0:1789760705.656216:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760705.656335:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495724032 00000100:00100000:2.0:1789760705.656345:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495724032 00000100:00100000:2.0:1789760705.656400:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689863988350 time 689938975830 state 0x0 ref 1 00000100:00100000:2.0:1789760705.656427:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689939236246 time 689938975830 state 0x1 ref 1 00000100:00100000:2.0:1789760705.656459:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689939236246 time 689977175594 state 0x0 ref 1 00000100:00100000:2.0:1789760705.656471:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495724032:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760705.656523:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20684800, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa007 00010000:00010000:2.0:1789760705.656533:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545451e2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.656551:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:2.0:1789760705.656556:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.656563:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545451e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.656582:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545451e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760705.692914:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972367 is committed 00002000:00010000:1.0:1789760705.693073:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20684800 -> 20688896 00010000:00010000:1.0:1789760705.693090:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545451e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.693102:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545451e2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.693111:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545451e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.693118:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760705.693123:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545451e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.693133:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545451e2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.693153:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545451e2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.693160:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545451e2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760705.693251:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495724032:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36778us (37039us total) trans 4294972367 rc 0/0 00000100:00100000:1.0:1789760705.693261:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5079 00000100:00100000:1.0:1789760705.693272:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689939236246 time 689977175594 state 0x2 ref 0 00000100:00100000:1.0:1789760705.697205:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760705.697422:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495724160 00000100:00100000:2.0:1789760705.697431:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495724160 00000100:00100000:2.0:1789760705.697485:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689939236246 time 689977175594 state 0x0 ref 1 00000100:00100000:2.0:1789760705.697500:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689977436010 time 689977175594 state 0x1 ref 1 00000100:00100000:2.0:1789760705.697530:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689977436010 time 690018246708 state 0x0 ref 1 00000100:00100000:2.0:1789760705.697545:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495724160:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760705.697619:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20688896, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa007 00010000:00010000:2.0:1789760705.697641:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545451e9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.697666:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:2.0:1789760705.697673:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.697685:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545451e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.697707:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545451e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760705.736527:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972368 is committed 00002000:00010000:3.0:1789760705.736706:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20688896 -> 20692992 00010000:00010000:3.0:1789760705.736722:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545451e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.736734:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545451e9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.736743:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545451e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.736750:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760705.736754:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545451e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.736763:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545451e9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.736849:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545451e9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.736858:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545451e9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760705.736976:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495724160:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39432us (39779us total) trans 4294972368 rc 0/0 00000100:00100000:3.0:1789760705.736985:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5080 00000100:00100000:3.0:1789760705.736994:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689977436010 time 690018246708 state 0x2 ref 0 00000100:00100000:0.0:1789760705.742123:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760705.742238:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495724288 00000100:00100000:2.0:1789760705.742249:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495724288 00000100:00100000:2.0:1789760705.742513:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 689977436010 time 690018246708 state 0x0 ref 1 00000100:00100000:2.0:1789760705.742527:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690018507124 time 690018246708 state 0x1 ref 1 00000100:00100000:2.0:1789760705.742585:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690018507124 time 690063301147 state 0x0 ref 1 00000100:00100000:2.0:1789760705.742598:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495724288:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760705.742652:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20692992, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa007 00010000:00010000:2.0:1789760705.742661:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545451f0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.742677:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:2.0:1789760705.742682:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.742689:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545451f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.742702:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545451f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760705.782151:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972369 is committed 00002000:00010000:1.0:1789760705.782608:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20692992 -> 20697088 00010000:00010000:1.0:1789760705.782625:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545451f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.782638:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545451f0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.782648:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545451f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.782656:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760705.782660:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545451f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.782666:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545451f0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.782685:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545451f0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.782693:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545451f0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760705.782819:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495724288:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40219us (40778us total) trans 4294972369 rc 0/0 00000100:00100000:1.0:1789760705.782830:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5081 00000100:00100000:1.0:1789760705.782839:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690018507124 time 690063301147 state 0x2 ref 0 00000100:00100000:1.0:1789760705.794619:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760705.794852:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495724416 00000100:00100000:2.0:1789760705.794863:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495724416 00000100:00100000:2.0:1789760705.794915:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690018507124 time 690063301147 state 0x0 ref 1 00000100:00100000:2.0:1789760705.794930:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690063561563 time 690063301147 state 0x1 ref 1 00000100:00100000:2.0:1789760705.794997:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690063561563 time 690115712587 state 0x0 ref 1 00000100:00100000:2.0:1789760705.795013:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370bd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495724416:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760705.795062:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20697088, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa007 00010000:00010000:2.0:1789760705.795072:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545451f7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.795088:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:2.0:1789760705.795092:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.795098:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545451f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.795112:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545451f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760705.831038:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972370 is committed 00002000:00010000:3.0:1789760705.831101:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20697088 -> 20701184 00010000:00010000:3.0:1789760705.831116:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545451f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.831131:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545451f7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.831142:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545451f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.831149:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760705.831155:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545451f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.831165:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545451f7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.831188:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545451f7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.831196:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545451f7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760705.831346:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370bd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495724416:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36322us (36724us total) trans 4294972370 rc 0/0 00000100:00100000:3.0:1789760705.831357:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5082 00000100:00100000:3.0:1789760705.831367:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690063561563 time 690115712587 state 0x2 ref 0 00000100:00100000:0.0:1789760705.835887:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760705.836236:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495724544 00000100:00100000:3.0:1789760705.836246:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495724544 00000100:00100000:3.0:1789760705.836339:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690063561563 time 690115712587 state 0x0 ref 1 00000100:00100000:3.0:1789760705.836360:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690115973003 time 690115712587 state 0x1 ref 1 00000100:00100000:3.0:1789760705.836395:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690115973003 time 690157111336 state 0x0 ref 1 00000100:00100000:3.0:1789760705.836409:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495724544:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760705.836455:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20701184, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa007 00010000:00010000:3.0:1789760705.836465:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545451fe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.836482:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760705.836487:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760705.836493:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545451fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.836507:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545451fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760705.881792:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20701184 -> 20705280 00010000:00010000:3.0:1789760705.881812:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545451fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.881826:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545451fe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.881836:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545451fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.881843:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760705.881848:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545451fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.881857:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545451fe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.881876:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545451fe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.881883:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545451fe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760705.881980:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495724544:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45570us (46098us total) trans 4294972371 rc 0/0 00000100:00100000:3.0:1789760705.881990:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5083 00000100:00100000:3.0:1789760705.882001:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690115973003 time 690157111336 state 0x2 ref 0 00000001:00080000:3.0:1789760705.882045:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972371 is committed 00000100:00100000:2.0:1789760705.885899:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760705.885998:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495724672 00000100:00100000:1.0:1789760705.886007:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495724672 00000100:00100000:1.0:1789760705.886058:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690115973003 time 690157111336 state 0x0 ref 1 00000100:00100000:1.0:1789760705.886073:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690157371752 time 690157111336 state 0x1 ref 1 00000100:00100000:1.0:1789760705.886105:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690157371752 time 690206821946 state 0x0 ref 1 00000100:00100000:1.0:1789760705.886116:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495724672:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760705.886159:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20705280, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa007 00010000:00010000:1.0:1789760705.886168:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545205 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.886185:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:1.0:1789760705.886189:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760705.886195:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545205 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.886209:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545205 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760705.929915:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972372 is committed 00002000:00010000:1.0:1789760705.930052:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20705280 -> 20709376 00010000:00010000:1.0:1789760705.930067:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545205 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.930080:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545205 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.930089:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545205 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.930096:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760705.930101:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545205 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.930110:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545205 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.930129:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545205 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.930137:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545205 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760705.930225:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495724672:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44106us (44330us total) trans 4294972372 rc 0/0 00000100:00100000:1.0:1789760705.930235:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5084 00000100:00100000:1.0:1789760705.930244:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690157371752 time 690206821946 state 0x2 ref 0 00000100:00100000:1.0:1789760705.933177:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760705.933248:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495724800 00000100:00100000:3.0:1789760705.933257:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495724800 00000100:00100000:3.0:1789760705.933342:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690157371752 time 690206821946 state 0x0 ref 1 00000100:00100000:3.0:1789760705.933357:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690207082362 time 690206821946 state 0x1 ref 1 00000100:00100000:3.0:1789760705.933385:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690207082362 time 690254102586 state 0x0 ref 1 00000100:00100000:3.0:1789760705.933397:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495724800:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760705.933439:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20709376, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa007 00010000:00010000:3.0:1789760705.933447:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454520c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.933462:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760705.933466:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760705.933471:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454520c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.933485:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454520c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760705.979348:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972373 is committed 00002000:00010000:3.0:1789760705.979472:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20709376 -> 20713472 00010000:00010000:3.0:1789760705.979487:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454520c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.979511:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454520c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.979522:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454520c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.979529:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760705.979534:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454520c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.979544:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454520c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.979563:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454520c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.979571:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454520c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760705.979677:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495724800:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46278us (46504us total) trans 4294972373 rc 0/0 00000100:00100000:3.0:1789760705.979687:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5085 00000100:00100000:3.0:1789760705.979698:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690207082362 time 690254102586 state 0x2 ref 0 00000100:00100000:0.0:1789760705.985172:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760705.985682:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495724928 00000100:00100000:3.0:1789760705.985693:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495724928 00000100:00100000:3.0:1789760705.985758:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690207082362 time 690254102586 state 0x0 ref 1 00000100:00100000:3.0:1789760705.985774:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690254363002 time 690254102586 state 0x1 ref 1 00000100:00100000:3.0:1789760705.985809:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690254363002 time 690306525889 state 0x0 ref 1 00000100:00100000:3.0:1789760705.985822:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495724928:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760705.985869:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20713472, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa007 00010000:00010000:3.0:1789760705.985878:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545213 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.985893:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760705.985899:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760705.985904:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545213 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.985917:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545213 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760706.030599:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972374 is committed 00002000:00010000:0.0:1789760706.030772:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20713472 -> 20717568 00010000:00010000:0.0:1789760706.030785:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545213 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.030795:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545213 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.030806:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545213 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.030811:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760706.030814:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545213 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.030820:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545213 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.030835:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545213 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.030840:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545213 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760706.030915:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495724928:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45091us (45748us total) trans 4294972374 rc 0/0 00000100:00100000:0.0:1789760706.030921:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5086 00000100:00100000:0.0:1789760706.030929:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690254363002 time 690306525889 state 0x2 ref 0 00000100:00100000:0.0:1789760706.036137:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760706.036226:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495725056 00000100:00100000:1.0:1789760706.036235:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495725056 00000100:00100000:1.0:1789760706.036471:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690254363002 time 690306525889 state 0x0 ref 1 00000100:00100000:1.0:1789760706.036487:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690306786305 time 690306525889 state 0x1 ref 1 00000100:00100000:2.0:1789760706.036646:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690306786305 time 690357359720 state 0x0 ref 1 00000100:00100000:2.0:1789760706.036662:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495725056:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760706.036728:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20717568, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa007 00010000:00010000:2.0:1789760706.036738:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454521a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.036756:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:2.0:1789760706.036761:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760706.036767:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454521a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.036789:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454521a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760706.060816:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972375 is committed 00002000:00010000:0.0:1789760706.061015:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20717568 -> 20721664 00010000:00010000:0.0:1789760706.061028:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454521a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.061045:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454521a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.061054:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454521a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.061060:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760706.061066:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454521a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.061074:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454521a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.061094:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454521a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.061102:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454521a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760706.061194:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495725056:12345-192.168.203.9@tcp:4:dd.0 Request processed in 24531us (25061us total) trans 4294972375 rc 0/0 00000100:00100000:0.0:1789760706.061204:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5087 00000100:00100000:0.0:1789760706.061214:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690306786305 time 690357359720 state 0x2 ref 0 00000100:00100000:3.0:1789760706.064812:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760706.064882:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495725184 00000100:00100000:2.0:1789760706.064893:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495725184 00000100:00100000:2.0:1789760706.064944:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690306786305 time 690357359720 state 0x0 ref 1 00000100:00100000:2.0:1789760706.064956:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690357620136 time 690357359720 state 0x1 ref 1 00000100:00100000:2.0:1789760706.064985:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690357620136 time 690385701834 state 0x0 ref 1 00000100:00100000:2.0:1789760706.064997:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495725184:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760706.065045:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20721664, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa007 00010000:00010000:2.0:1789760706.065054:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545221 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.065070:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:2.0:1789760706.065075:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760706.065081:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545221 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.065095:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545221 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760706.109671:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972376 is committed 00002000:00010000:2.0:1789760706.110088:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20721664 -> 20725760 00010000:00010000:2.0:1789760706.110105:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545221 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.110118:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545221 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.110126:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545221 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.110133:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760706.110138:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545221 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.110148:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545221 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.110170:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545221 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.110178:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545221 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760706.110625:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495725184:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45624us (45818us total) trans 4294972376 rc 0/0 00000100:00100000:2.0:1789760706.110637:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5088 00000100:00100000:2.0:1789760706.110648:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690357620136 time 690385701834 state 0x2 ref 0 00000100:00100000:2.0:1789760706.116901:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760706.117074:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495725312 00000100:00100000:0.0:1789760706.117084:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495725312 00000100:00100000:0.0:1789760706.117137:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690357620136 time 690385701834 state 0x0 ref 1 00000100:00100000:0.0:1789760706.117153:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690385962250 time 690385701834 state 0x1 ref 1 00000100:00100000:0.0:1789760706.117186:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690385962250 time 690437902653 state 0x0 ref 1 00000100:00100000:0.0:1789760706.117198:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495725312:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760706.117245:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20725760, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa007 00010000:00010000:0.0:1789760706.117253:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545228 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.117269:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1789760706.117274:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760706.117393:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545228 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.117410:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545228 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760706.147219:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20725760 -> 20729856 00010000:00010000:0.0:1789760706.147238:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545228 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.147250:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545228 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.147266:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545228 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.147274:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760706.147320:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545228 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.147331:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545228 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.147350:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545228 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.147358:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545228 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760706.147433:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495725312:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30234us (30539us total) trans 4294972377 rc 0/0 00000100:00100000:0.0:1789760706.147444:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5089 00000100:00100000:0.0:1789760706.147454:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690385962250 time 690437902653 state 0x2 ref 0 00000001:00080000:2.0:1789760706.147655:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972377 is committed 00000100:00100000:1.0:1789760706.150777:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760706.151017:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495725440 00000100:00100000:0.0:1789760706.151027:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495725440 00000100:00100000:0.0:1789760706.151078:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690385962250 time 690437902653 state 0x0 ref 1 00000100:00100000:0.0:1789760706.151093:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690438163069 time 690437902653 state 0x1 ref 1 00000100:00100000:0.0:1789760706.151151:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690438163069 time 690471868163 state 0x0 ref 1 00000100:00100000:0.0:1789760706.151164:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495725440:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760706.151208:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20729856, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa007 00010000:00010000:0.0:1789760706.151217:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454522f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.151232:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1789760706.151236:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760706.151242:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454522f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.151255:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454522f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760706.174266:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972378 is committed 00002000:00010000:0.0:1789760706.174448:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20729856 -> 20733952 00010000:00010000:0.0:1789760706.174463:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454522f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.174476:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454522f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.174485:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454522f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.174493:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760706.174499:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454522f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.174508:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454522f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.174528:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454522f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.174536:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454522f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760706.174618:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495725440:12345-192.168.203.9@tcp:4:dd.0 Request processed in 23452us (23844us total) trans 4294972378 rc 0/0 00000100:00100000:3.0:1789760706.177491:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760706.177616:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495725568 00000100:00100000:2.0:1789760706.177633:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495725568 00000100:00100000:0.0:1789760706.177746:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5090 00000100:00100000:2.0:1789760706.177946:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690472128579 time 690471868163 state 0x1 ref 1 00000100:00100000:2.0:1789760706.177999:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690472128579 time 690498716104 state 0x0 ref 1 00000100:00100000:2.0:1789760706.178012:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495725568:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760706.178056:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20733952, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa007 00010000:00010000:2.0:1789760706.178064:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545236 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.178079:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:2.0:1789760706.178084:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760706.178090:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545236 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.178105:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545236 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760706.208623:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972379 is committed 00002000:00010000:3.0:1789760706.208770:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20733952 -> 20738048 00010000:00010000:3.0:1789760706.208785:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545236 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.208799:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545236 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.208808:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545236 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.208815:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760706.208820:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545236 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.208830:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545236 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.208848:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545236 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.208855:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545236 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760706.208940:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495725568:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30926us (31452us total) trans 4294972379 rc 0/0 00000100:00100000:3.0:1789760706.208951:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5091 00000100:00100000:3.0:1789760706.208962:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690472128579 time 690498716104 state 0x2 ref 0 00000100:00100000:2.0:1789760706.212814:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760706.212923:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495725696 00000100:00100000:0.0:1789760706.212933:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495725696 00000100:00100000:0.0:1789760706.212981:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690472128579 time 690498716104 state 0x0 ref 1 00000100:00100000:0.0:1789760706.212996:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690498976520 time 690498716104 state 0x1 ref 1 00000100:00100000:0.0:1789760706.213028:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690498976520 time 690533745270 state 0x0 ref 1 00000100:00100000:0.0:1789760706.213041:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495725696:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760706.213089:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20738048, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa007 00010000:00010000:0.0:1789760706.213097:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454523d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.213112:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1789760706.213116:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760706.213123:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454523d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.213135:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454523d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760706.233917:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972380 is committed 00002000:00010000:0.0:1789760706.234061:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20738048 -> 20742144 00010000:00010000:0.0:1789760706.234080:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454523d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.234095:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454523d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.234104:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454523d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.234112:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760706.234117:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454523d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.234127:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454523d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.234147:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454523d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.234156:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454523d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760706.234277:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495725696:12345-192.168.203.9@tcp:4:dd.0 Request processed in 21234us (21467us total) trans 4294972380 rc 0/0 00000100:00100000:0.0:1789760706.234331:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5092 00000100:00100000:0.0:1789760706.234341:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690498976520 time 690533745270 state 0x2 ref 0 00000100:00100000:3.0:1789760706.237656:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760706.237707:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495725824 00000100:00100000:0.0:1789760706.237716:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495725824 00000100:00100000:0.0:1789760706.237763:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690498976520 time 690533745270 state 0x0 ref 1 00000100:00100000:0.0:1789760706.237778:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690534005686 time 690533745270 state 0x1 ref 1 00000100:00100000:0.0:1789760706.237807:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690534005686 time 690558524469 state 0x0 ref 1 00000100:00100000:0.0:1789760706.237819:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495725824:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760706.237863:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20742144, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa007 00010000:00010000:0.0:1789760706.237871:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545244 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.237886:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1789760706.237890:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760706.237896:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545244 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.237915:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545244 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760706.265094:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972381 is committed 00002000:00010000:0.0:1789760706.265174:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20742144 -> 20746240 00010000:00010000:0.0:1789760706.265188:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545244 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.265201:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545244 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.265211:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545244 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.265219:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760706.265224:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545244 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.265234:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545244 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.265252:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545244 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.265261:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545244 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760706.265460:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495725824:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27638us (27809us total) trans 4294972381 rc 0/0 00000100:00100000:0.0:1789760706.265471:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5093 00000100:00100000:0.0:1789760706.265480:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690534005686 time 690558524469 state 0x2 ref 0 00000100:00100000:2.0:1789760706.269615:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760706.269732:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495725952 00000100:00100000:3.0:1789760706.269742:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495725952 00000100:00100000:3.0:1789760706.269790:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690534005686 time 690558524469 state 0x0 ref 1 00000100:00100000:3.0:1789760706.269805:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690558784885 time 690558524469 state 0x1 ref 1 00000100:00100000:3.0:1789760706.269866:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690558784885 time 690590582099 state 0x0 ref 1 00000100:00100000:3.0:1789760706.269878:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495725952:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760706.269920:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20746240, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa007 00010000:00010000:3.0:1789760706.269929:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454524b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.269945:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760706.269948:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760706.269954:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454524b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.269967:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454524b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760706.296479:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20746240 -> 20750336 00010000:00010000:0.0:1789760706.296499:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454524b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.296512:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454524b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.296523:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454524b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.296530:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760706.296535:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454524b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.296544:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454524b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.296563:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454524b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.296577:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454524b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760706.296667:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495725952:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26785us (27055us total) trans 4294972382 rc 0/0 00000100:00100000:0.0:1789760706.296677:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5094 00000100:00100000:0.0:1789760706.296687:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690558784885 time 690590582099 state 0x2 ref 0 00000001:00080000:1.0:1789760706.296758:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972382 is committed 00000100:00100000:2.0:1789760706.299736:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760706.299840:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495726080 00000100:00100000:1.0:1789760706.299851:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495726080 00000100:00100000:1.0:1789760706.299901:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690558784885 time 690590582099 state 0x0 ref 1 00000100:00100000:1.0:1789760706.299915:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690590842515 time 690590582099 state 0x1 ref 1 00000100:00100000:1.0:1789760706.299945:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690590842515 time 690620662449 state 0x0 ref 1 00000100:00100000:1.0:1789760706.299956:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495726080:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760706.299998:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20750336, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa007 00010000:00010000:1.0:1789760706.300006:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545252 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.300022:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:1.0:1789760706.300026:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760706.300032:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545252 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.300045:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545252 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760706.324335:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20750336 -> 20754432 00010000:00010000:1.0:1789760706.324353:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545252 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.324367:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545252 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.324376:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545252 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.324391:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760706.324396:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545252 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.324407:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545252 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.324424:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545252 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.324434:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545252 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760706.324511:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495726080:12345-192.168.203.9@tcp:4:dd.0 Request processed in 24550us (24777us total) trans 4294972383 rc 0/0 00000100:00100000:1.0:1789760706.324521:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5095 00000100:00100000:1.0:1789760706.324531:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690590842515 time 690620662449 state 0x2 ref 0 00000001:00080000:0.0:1789760706.327333:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972383 is committed 00000100:00100000:2.0:1789760706.328891:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760706.328983:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495726208 00000100:00100000:0.0:1789760706.328992:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495726208 00000100:00100000:0.0:1789760706.329037:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690590842515 time 690620662449 state 0x0 ref 1 00000100:00100000:0.0:1789760706.329052:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690620922865 time 690620662449 state 0x1 ref 1 00000100:00100000:0.0:1789760706.329108:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690620922865 time 690649824236 state 0x0 ref 1 00000100:00100000:0.0:1789760706.329121:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495726208:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760706.329164:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20754432, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa007 00010000:00010000:0.0:1789760706.329173:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545259 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.329189:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1789760706.329193:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760706.329199:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545259 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.329214:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545259 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760706.383501:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972384 is committed 00002000:00010000:3.0:1789760706.383549:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20754432 -> 20758528 00010000:00010000:3.0:1789760706.383565:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545259 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.383578:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545259 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.383587:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545259 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.383594:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760706.383599:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545259 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.383608:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545259 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.383635:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545259 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.383643:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545259 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760706.383726:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495726208:12345-192.168.203.9@tcp:4:dd.0 Request processed in 54602us (54836us total) trans 4294972384 rc 0/0 00000100:00100000:3.0:1789760706.383851:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5096 00000100:00100000:3.0:1789760706.383862:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690620922865 time 690649824236 state 0x2 ref 0 00000100:00100000:1.0:1789760706.387274:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760706.387487:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495726336 00000100:00100000:3.0:1789760706.387497:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495726336 00000100:00100000:3.0:1789760706.387600:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690620922865 time 690649824236 state 0x0 ref 1 00000100:00100000:3.0:1789760706.387614:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690650084652 time 690649824236 state 0x1 ref 1 00000100:00100000:3.0:1789760706.387644:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690650084652 time 690708360933 state 0x0 ref 1 00000100:00100000:3.0:1789760706.387655:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495726336:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760706.387696:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20758528, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa007 00010000:00010000:3.0:1789760706.387704:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545260 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.387719:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760706.387723:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760706.387792:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545260 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.387807:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545260 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.400454:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760706.422022:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972385 is committed 00002000:00010000:3.0:1789760706.422209:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20758528 -> 20762624 00010000:00010000:3.0:1789760706.422225:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545260 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.422239:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545260 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.422248:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545260 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.422256:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760706.422261:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545260 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.422270:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545260 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.422332:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545260 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.422340:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545260 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760706.422531:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495726336:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34871us (35259us total) trans 4294972385 rc 0/0 00000100:00100000:3.0:1789760706.422543:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5097 00000100:00100000:3.0:1789760706.422553:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690650084652 time 690708360933 state 0x2 ref 0 00000100:00100000:2.0:1789760706.431955:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760706.432164:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495726464 00000100:00100000:1.0:1789760706.432175:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495726464 00000100:00100000:1.0:1789760706.432224:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690650084652 time 690708360933 state 0x0 ref 1 00000100:00100000:1.0:1789760706.432238:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690708621349 time 690708360933 state 0x1 ref 1 00000100:00100000:1.0:1789760706.432273:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690708621349 time 690752989610 state 0x0 ref 1 00000100:00100000:1.0:1789760706.432561:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903639dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495726464:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760706.432610:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20762624, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa007 00010000:00010000:1.0:1789760706.432620:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545267 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.432647:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:1.0:1789760706.432652:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760706.432659:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545267 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.432673:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545267 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760706.470926:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20762624 -> 20766720 00010000:00010000:1.0:1789760706.470945:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545267 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.470960:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545267 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.470968:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545267 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.470976:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760706.470981:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545267 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.470989:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545267 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.471010:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545267 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.471018:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545267 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760706.471120:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903639dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495726464:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38561us (39170us total) trans 4294972386 rc 0/0 00000100:00100000:1.0:1789760706.471131:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5098 00000100:00100000:1.0:1789760706.471141:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690708621349 time 690752989610 state 0x2 ref 0 00000001:00080000:3.0:1789760706.473097:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972386 is committed 00000100:00100000:2.0:1789760706.475930:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760706.476196:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495726592 00000100:00100000:3.0:1789760706.476206:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495726592 00000100:00100000:3.0:1789760706.476256:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690708621349 time 690752989610 state 0x0 ref 1 00000100:00100000:3.0:1789760706.476272:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690753250026 time 690752989610 state 0x1 ref 1 00000100:00100000:3.0:1789760706.476333:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690753250026 time 690797049000 state 0x0 ref 1 00000100:00100000:3.0:1789760706.476346:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495726592:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760706.476401:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20766720, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa007 00010000:00010000:3.0:1789760706.476410:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454526e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.476424:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760706.476429:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760706.476436:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454526e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.476449:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454526e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760706.537806:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972387 is committed 00002000:00010000:3.0:1789760706.538075:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20766720 -> 20770816 00010000:00010000:3.0:1789760706.538091:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454526e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.538102:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454526e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.538116:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454526e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.538123:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760706.538128:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454526e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.538137:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454526e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.538157:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454526e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.538164:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454526e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760706.541159:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495726592:12345-192.168.203.9@tcp:4:dd.0 Request processed in 64801us (65224us total) trans 4294972387 rc 0/0 00000100:00100000:3.0:1789760706.541176:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5099 00000100:00100000:3.0:1789760706.541191:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690753250026 time 690797049000 state 0x2 ref 0 00000100:00100000:0.0:1789760706.544818:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760706.544953:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495726720 00000100:00100000:3.0:1789760706.544963:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495726720 00000100:00100000:3.0:1789760706.545016:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690753250026 time 690797049000 state 0x0 ref 1 00000100:00100000:3.0:1789760706.545032:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690797309416 time 690797049000 state 0x1 ref 1 00000100:00100000:3.0:1789760706.545172:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690797309416 time 690865888166 state 0x0 ref 1 00000100:00100000:3.0:1789760706.545186:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495726720:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760706.545234:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20770816, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa007 00010000:00010000:3.0:1789760706.545243:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545275 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.545259:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760706.545263:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760706.545270:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545275 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.545339:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545275 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760706.579930:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972388 is committed 00002000:00010000:3.0:1789760706.580846:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20770816 -> 20774912 00010000:00010000:3.0:1789760706.580862:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545275 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.580875:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545275 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.580884:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545275 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.580892:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760706.580897:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545275 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.580906:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545275 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.580928:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545275 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.580935:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545275 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760706.581078:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495726720:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35889us (36264us total) trans 4294972388 rc 0/0 00000100:00100000:3.0:1789760706.581091:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5100 00000100:00100000:3.0:1789760706.581101:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690797309416 time 690865888166 state 0x2 ref 0 00000100:00100000:2.0:1789760706.589873:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760706.589966:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495726848 00000100:00100000:1.0:1789760706.589976:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495726848 00000100:00100000:1.0:1789760706.590028:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690797309416 time 690865888166 state 0x0 ref 1 00000100:00100000:1.0:1789760706.590044:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690866148582 time 690865888166 state 0x1 ref 1 00000100:00100000:1.0:1789760706.590073:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690866148582 time 690910790330 state 0x0 ref 1 00000100:00100000:1.0:1789760706.590086:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495726848:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760706.590133:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20774912, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa007 00010000:00010000:1.0:1789760706.590142:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454527c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.590158:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:1.0:1789760706.590162:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760706.590169:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454527c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.590183:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454527c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760706.651930:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972389 is committed 00002000:00010000:0.0:1789760706.657603:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20774912 -> 20779008 00010000:00010000:0.0:1789760706.657638:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454527c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.657653:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454527c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.657662:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454527c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.657669:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760706.657674:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454527c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.657683:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454527c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.657703:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454527c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.657715:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454527c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760706.657830:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495726848:12345-192.168.203.9@tcp:4:dd.0 Request processed in 67741us (67962us total) trans 4294972389 rc 0/0 00000100:00100000:0.0:1789760706.657841:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5101 00000100:00100000:0.0:1789760706.657852:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690866148582 time 690910790330 state 0x2 ref 0 00000100:00100000:2.0:1789760706.662266:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760706.662441:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495726976 00000100:00100000:3.0:1789760706.662452:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495726976 00000100:00100000:3.0:1789760706.662508:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690866148582 time 690910790330 state 0x0 ref 1 00000100:00100000:3.0:1789760706.662523:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690911050746 time 690910790330 state 0x1 ref 1 00000100:00100000:3.0:1789760706.662552:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690911050746 time 690983269093 state 0x0 ref 1 00000100:00100000:3.0:1789760706.662564:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495726976:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760706.662609:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20779008, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa007 00010000:00010000:3.0:1789760706.662619:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545283 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.662635:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760706.662639:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760706.662645:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545283 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.662657:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545283 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760706.688479:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972390 is committed 00002000:00010000:2.0:1789760706.688752:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20779008 -> 20783104 00002000:00010000:2.0:1789760706.688757:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760705 -> 1789760706 00002000:00010000:2.0:1789760706.688760:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760705 -> 1789760706 00010000:00010000:2.0:1789760706.688774:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545283 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.688787:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545283 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.688796:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545283 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.688804:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760706.688809:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545283 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.688819:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545283 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.688842:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545283 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.688854:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545283 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760706.688951:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495726976:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26382us (26689us total) trans 4294972390 rc 0/0 00000100:00100000:2.0:1789760706.688962:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5102 00000100:00100000:2.0:1789760706.688972:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690911050746 time 690983269093 state 0x2 ref 0 00000100:00100000:1.0:1789760706.695937:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760706.696017:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495727104 00000100:00100000:3.0:1789760706.696026:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495727104 00000100:00100000:3.0:1789760706.696077:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690911050746 time 690983269093 state 0x0 ref 1 00000100:00100000:3.0:1789760706.696092:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690983529509 time 690983269093 state 0x1 ref 1 00000100:00100000:3.0:1789760706.696121:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690983529509 time 691016837525 state 0x0 ref 1 00000100:00100000:3.0:1789760706.696140:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495727104:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760706.696182:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20783104, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xa007 00010000:00010000:3.0:1789760706.696191:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454528a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.696206:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760706.696210:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760706.696216:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454528a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.696230:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454528a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760706.730602:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972391 is committed 00002000:00010000:3.0:1789760706.730793:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20783104 -> 20787200 00010000:00010000:3.0:1789760706.730809:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454528a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.730822:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454528a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.730832:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454528a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.730839:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760706.730845:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454528a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.730853:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454528a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.730872:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454528a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.730879:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454528a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760706.730977:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495727104:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34834us (35047us total) trans 4294972391 rc 0/0 00000100:00100000:3.0:1789760706.730987:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5103 00000100:00100000:3.0:1789760706.730997:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690983529509 time 691016837525 state 0x2 ref 0 00000100:00100000:3.0:1789760706.734242:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760706.734590:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495727232 00000100:00100000:1.0:1789760706.734600:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495727232 00000100:00100000:1.0:1789760706.734663:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 690983529509 time 691016837525 state 0x0 ref 1 00000100:00100000:1.0:1789760706.734678:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691017097941 time 691016837525 state 0x1 ref 1 00000100:00100000:1.0:1789760706.734705:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691017097941 time 691055422213 state 0x0 ref 1 00000100:00100000:1.0:1789760706.734717:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495727232:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760706.734761:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20787200, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xa007 00010000:00010000:1.0:1789760706.734770:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545291 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.734786:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:1.0:1789760706.734790:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760706.734797:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545291 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.734811:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545291 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760706.783967:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972392 is committed 00002000:00010000:3.0:1789760706.784855:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20787200 -> 20791296 00010000:00010000:3.0:1789760706.784869:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545291 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.784884:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545291 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.784894:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545291 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.784901:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760706.784906:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545291 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.784916:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545291 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.784934:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545291 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.784948:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545291 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760706.785044:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495727232:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50322us (50809us total) trans 4294972392 rc 0/0 00000100:00100000:3.0:1789760706.785053:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5104 00000100:00100000:3.0:1789760706.785064:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691017097941 time 691055422213 state 0x2 ref 0 00000100:00100000:1.0:1789760706.792950:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760706.793971:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495727360 00000100:00100000:3.0:1789760706.793982:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495727360 00000100:00100000:3.0:1789760706.794042:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691017097941 time 691055422213 state 0x0 ref 1 00000100:00100000:3.0:1789760706.794058:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691055682629 time 691055422213 state 0x1 ref 1 00000100:00100000:3.0:1789760706.794190:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691055682629 time 691114905935 state 0x0 ref 1 00000100:00100000:3.0:1789760706.794202:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495727360:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760706.794252:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20791296, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xa007 00010000:00010000:3.0:1789760706.794262:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545298 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.794327:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760706.794333:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760706.794339:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545298 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.794360:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545298 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760706.828566:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20791296 -> 20795392 00010000:00010000:3.0:1789760706.828585:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545298 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.828609:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545298 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.828619:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545298 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.828740:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760706.828745:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545298 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.828755:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545298 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.828773:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545298 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.828781:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545298 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760706.828906:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495727360:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34701us (35961us total) trans 4294972393 rc 0/0 00000100:00100000:3.0:1789760706.828917:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5105 00000100:00100000:3.0:1789760706.828926:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691055682629 time 691114905935 state 0x2 ref 0 00000001:00080000:3.0:1789760706.828962:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972393 is committed 00000100:00100000:3.0:1789760706.832659:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760706.832728:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495727488 00000100:00100000:2.0:1789760706.832737:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495727488 00000100:00100000:2.0:1789760706.832786:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691055682629 time 691114905935 state 0x0 ref 1 00000100:00100000:2.0:1789760706.832800:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691115166351 time 691114905935 state 0x1 ref 1 00000100:00100000:2.0:1789760706.832832:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691115166351 time 691153548916 state 0x0 ref 1 00000100:00100000:2.0:1789760706.832844:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495727488:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760706.832890:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20795392, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xa007 00010000:00010000:2.0:1789760706.832898:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454529f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.832914:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:2.0:1789760706.832918:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760706.832924:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454529f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.832941:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454529f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760706.846789:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f02769180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3318:1876699520300032:0@lo:400:kworker.0 00000100:00100000:2.0:1789760706.846853:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760706.846903:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f01ac6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3318:1876699520300544:0@lo:400:kworker.0 00000100:00100000:2.0:1789760706.846925:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760706.846953:0:3318:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1789760706.846958:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f0276a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:59c4e264-9b75-482b-8813-3d1dc124f384:3318:1876699520299904:0@lo:400:kworker.0 00000100:00100000:2.0:1789760706.846981:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760706.847004:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f01ac6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3318:1876699520300416:0@lo:400:kworker.0 00000100:00100000:2.0:1789760706.847025:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760706.847051:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f0276b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520300160:0@lo:400:kworker.0 00000100:00100000:2.0:1789760706.847075:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760706.847094:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789760706.847099:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f02769880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520300288:0@lo:400:kworker.0 00000100:00100000:2.0:1789760706.847121:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760706.847154:0:8003:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520300032 00000100:00100000:2.0:1789760706.847160:0:8003:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520300032 00000100:00100000:2.0:1789760706.847195:0:8003:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520300416 00000100:00100000:2.0:1789760706.847197:0:8003:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520300416 00000100:00100000:2.0:1789760706.847218:0:8003:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520300160 00000100:00100000:2.0:1789760706.847220:0:8003:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520300160 00000100:00100000:2.0:1789760706.847234:0:8003:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 447 00000100:00100000:2.0:1789760706.847241:0:8003:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3318:x1876699520300032:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760706.847327:0:8003:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3318:x1876699520300032:12345-0@lo:400:kworker.0 Request processed in 85us (473us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760706.847335:0:8003:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 447 00000100:00100000:2.0:1789760706.847350:0:8003:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 448 00000100:00100000:2.0:1789760706.847356:0:8003:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1876699520300416:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760706.847379:0:8003:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1876699520300416:12345-0@lo:400:kworker.0 Request processed in 25us (354us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760706.847385:0:8003:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 448 00000100:00100000:2.0:1789760706.847393:0:8003:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 449 00000100:00100000:2.0:1789760706.847398:0:8003:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520300160:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760706.847422:0:8003:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520300160:12345-0@lo:400:kworker.0 Request processed in 24us (347us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760706.847428:0:8003:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 449 00000100:00100000:2.0:1789760706.847442:0:13058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520300544 00000100:00100000:2.0:1789760706.847445:0:13058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520300544 00000100:00100000:2.0:1789760706.847467:0:13058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520300288 00000100:00100000:2.0:1789760706.847469:0:13058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520300288 00000100:00100000:2.0:1789760706.847485:0:13058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 210 00000100:00100000:2.0:1789760706.847491:0:13058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3318:x1876699520300544:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760706.847521:0:13058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3318:x1876699520300544:12345-0@lo:400:kworker.0 Request processed in 31us (595us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760706.847527:0:13058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 210 00000100:00100000:2.0:1789760706.847536:0:13058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 211 00000100:00100000:2.0:1789760706.847541:0:13058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520300288:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760706.847561:0:13058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520300288:12345-0@lo:400:kworker.0 Request processed in 21us (439us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760706.847569:0:13058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 211 00000100:00100000:2.0:1789760706.847583:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520299904 00000100:00100000:2.0:1789760706.847586:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520299904 00000100:00100000:2.0:1789760706.847607:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 387 00000100:00100000:2.0:1789760706.847612:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929012316450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:59c4e264-9b75-482b-8813-3d1dc124f384+19:3318:x1876699520299904:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760706.847635:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929012316450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:59c4e264-9b75-482b-8813-3d1dc124f384+19:3318:x1876699520299904:12345-0@lo:400:kworker.0 Request processed in 23us (654us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760706.847641:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 387 00000100:00080000:2.0:1789760706.847674:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789760706.847677:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760706.847679:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760706.847713:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f02769880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520300288:0@lo:400:kworker.0 00000100:00100000:2.0:1789760706.847723:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f0276b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520300160:0@lo:400:kworker.0 00000100:00100000:2.0:1789760706.847747:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f0276a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:59c4e264-9b75-482b-8813-3d1dc124f384:3318:1876699520299904:0@lo:400:kworker.0 00000100:00100000:2.0:1789760706.847756:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f01ac6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3318:1876699520300416:0@lo:400:kworker.0 00000100:00100000:2.0:1789760706.847764:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f02769180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3318:1876699520300032:0@lo:400:kworker.0 00000100:00100000:2.0:1789760706.847771:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f01ac6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3318:1876699520300544:0@lo:400:kworker.0 00000001:00080000:1.0:1789760706.858924:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972394 is committed 00002000:00010000:3.0:1789760706.859197:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20795392 -> 20799488 00010000:00010000:3.0:1789760706.859213:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454529f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.859225:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454529f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.859234:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454529f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.859241:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760706.859246:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454529f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.859255:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454529f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.859273:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454529f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.859314:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454529f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760706.859398:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495727488:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26551us (26743us total) trans 4294972394 rc 0/0 00000100:00100000:3.0:1789760706.859408:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5106 00000100:00100000:3.0:1789760706.859418:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691115166351 time 691153548916 state 0x2 ref 0 00000100:00100000:1.0:1789760706.864382:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760706.864503:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495727616 00000100:00100000:3.0:1789760706.864513:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495727616 00000100:00100000:3.0:1789760706.864571:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691115166351 time 691153548916 state 0x0 ref 1 00000100:00100000:3.0:1789760706.864585:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691153809332 time 691153548916 state 0x1 ref 1 00000100:00100000:3.0:1789760706.864613:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691153809332 time 691185330170 state 0x0 ref 1 00000100:00100000:3.0:1789760706.864632:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495727616:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760706.864675:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20799488, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xa007 00010000:00010000:3.0:1789760706.864683:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452a6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.864699:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760706.864703:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760706.864709:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.864727:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760706.897420:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972395 is committed 00002000:00010000:1.0:1789760706.897891:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20799488 -> 20803584 00010000:00010000:1.0:1789760706.897906:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.897920:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452a6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.897931:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.897938:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760706.897944:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.897953:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452a6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.897972:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452a6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.897981:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452a6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760706.898071:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495727616:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33443us (33694us total) trans 4294972395 rc 0/0 00000100:00100000:1.0:1789760706.898085:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5107 00000100:00100000:1.0:1789760706.898095:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691153809332 time 691185330170 state 0x2 ref 0 00000100:00100000:1.0:1789760706.902100:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760706.902189:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495727744 00000100:00100000:3.0:1789760706.902199:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495727744 00000100:00100000:3.0:1789760706.902250:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691153809332 time 691185330170 state 0x0 ref 1 00000100:00100000:3.0:1789760706.902264:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691185590586 time 691185330170 state 0x1 ref 1 00000100:00100000:3.0:1789760706.902462:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691185590586 time 691223177637 state 0x0 ref 1 00000100:00100000:3.0:1789760706.902475:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495727744:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760706.902777:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20803584, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xa007 00010000:00010000:3.0:1789760706.902788:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545452ad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.902804:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760706.902808:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760706.902814:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545452ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.902827:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545452ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760706.935239:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20803584 -> 20807680 00010000:00010000:0.0:1789760706.935260:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545452ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.935275:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545452ad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.935328:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545452ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.935335:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760706.935340:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545452ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.935350:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545452ad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.935377:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545452ad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.935384:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545452ad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760706.935467:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495727744:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32990us (33373us total) trans 4294972396 rc 0/0 00000100:00100000:0.0:1789760706.935477:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5108 00000100:00100000:0.0:1789760706.935487:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691185590586 time 691223177637 state 0x2 ref 0 00000001:00080000:2.0:1789760706.939007:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972396 is committed 00000100:00100000:2.0:1789760706.941792:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760706.941869:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495727872 00000100:00100000:3.0:1789760706.941878:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495727872 00000100:00100000:3.0:1789760706.941927:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691185590586 time 691223177637 state 0x0 ref 1 00000100:00100000:3.0:1789760706.941942:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691223438053 time 691223177637 state 0x1 ref 1 00000100:00100000:3.0:1789760706.942015:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691223438053 time 691262731604 state 0x0 ref 1 00000100:00100000:3.0:1789760706.942028:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495727872:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760706.942075:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20807680, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xa007 00010000:00010000:3.0:1789760706.942084:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452b4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.942100:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760706.942105:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760706.942112:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.942127:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760706.999034:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972397 is committed 00002000:00010000:3.0:1789760707.004841:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20807680 -> 20811776 00010000:00010000:3.0:1789760707.004862:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.004876:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452b4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.004892:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.004900:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760707.004906:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.004916:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.004935:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452b4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.004943:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452b4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760707.005041:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495727872:12345-192.168.203.9@tcp:4:dd.0 Request processed in 63010us (63253us total) trans 4294972397 rc 0/0 00000100:00100000:3.0:1789760707.005051:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5109 00000100:00100000:3.0:1789760707.005061:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691223438053 time 691262731604 state 0x2 ref 0 00000100:00100000:1.0:1789760707.008471:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760707.008617:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495728000 00000100:00100000:3.0:1789760707.008638:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495728000 00000100:00100000:3.0:1789760707.008714:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691223438053 time 691262731604 state 0x0 ref 1 00000100:00100000:3.0:1789760707.008731:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691262992020 time 691262731604 state 0x1 ref 1 00000100:00100000:3.0:1789760707.008763:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691262992020 time 691329480324 state 0x0 ref 1 00000100:00100000:3.0:1789760707.008777:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495728000:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760707.008832:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20811776, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xa007 00010000:00010000:3.0:1789760707.008842:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452bb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.008857:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760707.008861:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760707.008867:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.008880:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760707.009712:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789760707.043138:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20811776 -> 20815872 00010000:00010000:3.0:1789760707.043158:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.043170:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452bb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.043179:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.043187:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760707.043193:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.043202:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452bb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.043222:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452bb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.043230:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452bb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760707.043420:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495728000:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34640us (34951us total) trans 4294972398 rc 0/0 00000100:00100000:3.0:1789760707.043432:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5110 00000100:00100000:3.0:1789760707.043441:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691262992020 time 691329480324 state 0x2 ref 0 00000001:00080000:0.0:1789760707.046740:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972398 is committed 00000100:00100000:1.0:1789760707.057844:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760707.057991:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495728128 00000100:00100000:0.0:1789760707.058001:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495728128 00000100:00100000:0.0:1789760707.058058:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691262992020 time 691329480324 state 0x0 ref 1 00000100:00100000:0.0:1789760707.058075:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691329740740 time 691329480324 state 0x1 ref 1 00000100:00100000:0.0:1789760707.058109:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691329740740 time 691378826010 state 0x0 ref 1 00000100:00100000:0.0:1789760707.058121:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495728128:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760707.058172:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20815872, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xa007 00010000:00010000:0.0:1789760707.058182:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545452c2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.058205:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:0.0:1789760707.058209:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760707.058216:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545452c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.058230:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545452c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760707.113986:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972399 is committed 00002000:00010000:1.0:1789760707.114237:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20815872 -> 20819968 00010000:00010000:1.0:1789760707.114253:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545452c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.114267:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545452c2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.114473:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545452c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.114484:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760707.114490:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545452c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.114498:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545452c2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.114580:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545452c2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.114594:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545452c2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760707.114881:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495728128:12345-192.168.203.9@tcp:4:dd.0 Request processed in 56755us (57040us total) trans 4294972399 rc 0/0 00000100:00100000:1.0:1789760707.114892:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5111 00000100:00100000:1.0:1789760707.114902:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691329740740 time 691378826010 state 0x2 ref 0 00000100:00100000:3.0:1789760707.122940:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760707.123050:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495728256 00000100:00100000:0.0:1789760707.123061:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495728256 00000100:00100000:0.0:1789760707.123115:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691329740740 time 691378826010 state 0x0 ref 1 00000100:00100000:0.0:1789760707.123130:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691379086426 time 691378826010 state 0x1 ref 1 00000100:00100000:0.0:1789760707.123161:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691379086426 time 691443877597 state 0x0 ref 1 00000100:00100000:0.0:1789760707.123174:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495728256:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760707.123232:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20819968, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xa007 00010000:00010000:0.0:1789760707.123241:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545452c9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.123258:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:0.0:1789760707.123263:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760707.123269:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545452c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.123453:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545452c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760707.174107:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972400 is committed 00002000:00010000:1.0:1789760707.174216:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20819968 -> 20824064 00010000:00010000:1.0:1789760707.174231:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545452c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.174246:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545452c9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.174256:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545452c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.174264:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760707.174269:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545452c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.174322:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545452c9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.174604:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545452c9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.174616:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545452c9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760707.175005:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495728256:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51827us (52069us total) trans 4294972400 rc 0/0 00000100:00100000:1.0:1789760707.175017:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5112 00000100:00100000:1.0:1789760707.175027:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691379086426 time 691443877597 state 0x2 ref 0 00000100:00100000:3.0:1789760707.183374:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760707.184007:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495728384 00000100:00100000:0.0:1789760707.184017:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495728384 00000100:00100000:0.0:1789760707.184151:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691379086426 time 691443877597 state 0x0 ref 1 00000100:00100000:0.0:1789760707.184167:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691444138013 time 691443877597 state 0x1 ref 1 00000100:00100000:0.0:1789760707.184202:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691444138013 time 691504918614 state 0x0 ref 1 00000100:00100000:0.0:1789760707.184215:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495728384:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760707.184262:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20824064, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xa007 00010000:00010000:0.0:1789760707.184272:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545452d0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.184372:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:0.0:1789760707.184376:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760707.184456:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545452d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.184472:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545452d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760707.227545:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972401 is committed 00002000:00010000:0.0:1789760707.227743:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20824064 -> 20828160 00010000:00010000:0.0:1789760707.227757:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545452d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.227770:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545452d0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.227779:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545452d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.227786:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760707.227791:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545452d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.227801:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545452d0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.227820:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545452d0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.227827:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545452d0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760707.227913:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495728384:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43696us (44545us total) trans 4294972401 rc 0/0 00000100:00100000:0.0:1789760707.227923:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5113 00000100:00100000:0.0:1789760707.227934:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691444138013 time 691504918614 state 0x2 ref 0 00000100:00100000:1.0:1789760707.237686:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760707.237851:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495728512 00000100:00100000:0.0:1789760707.237860:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495728512 00000100:00100000:0.0:1789760707.237914:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691444138013 time 691504918614 state 0x0 ref 1 00000100:00100000:0.0:1789760707.237931:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691505179030 time 691504918614 state 0x1 ref 1 00000100:00100000:0.0:1789760707.237968:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691505179030 time 691558685220 state 0x0 ref 1 00000100:00100000:0.0:1789760707.237981:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495728512:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760707.238035:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20828160, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xa007 00010000:00010000:0.0:1789760707.238045:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545452d7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.238062:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:0.0:1789760707.238068:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760707.238090:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545452d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.238103:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545452d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760707.308037:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972402 is committed 00002000:00010000:2.0:1789760707.309487:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20828160 -> 20832256 00010000:00010000:2.0:1789760707.309505:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545452d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.309518:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545452d7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.309528:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545452d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.309535:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760707.309540:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545452d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.309550:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545452d7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.309568:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545452d7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.309576:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545452d7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760707.309668:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495728512:12345-192.168.203.9@tcp:4:dd.0 Request processed in 71686us (71987us total) trans 4294972402 rc 0/0 00000100:00100000:2.0:1789760707.309678:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5114 00000100:00100000:2.0:1789760707.309688:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691505179030 time 691558685220 state 0x2 ref 0 00000100:00100000:3.0:1789760707.312878:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760707.312962:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495728640 00000100:00100000:1.0:1789760707.312971:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495728640 00000100:00100000:1.0:1789760707.313021:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691505179030 time 691558685220 state 0x0 ref 1 00000100:00100000:1.0:1789760707.313037:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691558945636 time 691558685220 state 0x1 ref 1 00000100:00100000:1.0:1789760707.313067:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691558945636 time 691633784531 state 0x0 ref 1 00000100:00100000:1.0:1789760707.313078:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495728640:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760707.313124:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20832256, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xa007 00010000:00010000:1.0:1789760707.313132:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545452de lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.313153:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:1.0:1789760707.313157:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760707.313163:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545452de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.313175:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545452de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760707.356965:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20832256 -> 20836352 00010000:00010000:3.0:1789760707.356985:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545452de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.356998:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545452de lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.357014:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545452de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.357022:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760707.357027:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545452de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.357035:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545452de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.357054:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545452de lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.357062:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545452de lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760707.357157:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495728640:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44075us (44282us total) trans 4294972403 rc 0/0 00000100:00100000:3.0:1789760707.357168:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5115 00000100:00100000:3.0:1789760707.357178:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691558945636 time 691633784531 state 0x2 ref 0 00000001:00080000:3.0:1789760707.360366:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972403 is committed 00000100:00100000:0.0:1789760707.362134:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760707.362913:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495728768 00000100:00100000:3.0:1789760707.362929:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495728768 00000100:00100000:3.0:1789760707.362981:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691558945636 time 691633784531 state 0x0 ref 1 00000100:00100000:3.0:1789760707.363000:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691634044947 time 691633784531 state 0x1 ref 1 00000100:00100000:3.0:1789760707.363021:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691634044947 time 691683738700 state 0x0 ref 1 00000100:00100000:3.0:1789760707.363033:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495728768:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760707.363077:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20836352, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xa007 00010000:00010000:3.0:1789760707.363090:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452e5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.363105:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760707.363110:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760707.363116:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.363129:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760707.367030:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f0492f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520300672:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760707.367100:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760707.367216:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520300672 00000100:00100000:3.0:1789760707.367223:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520300672 00000100:00100000:3.0:1789760707.367269:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 186 00000100:00100000:3.0:1789760707.367294:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520300672:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760707.367337:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760707.367378:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520300672:12345-0@lo:13:osp-pre-1-0.0 Request processed in 102us (277us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760707.367385:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 186 00000100:00100000:2.0:1789760707.367505:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f0492f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520300672:0@lo:13:osp-pre-1-0.0 00000001:00080000:0.0:1789760707.411584:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972404 is committed 00002000:00010000:3.0:1789760707.411786:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20836352 -> 20840448 00010000:00010000:3.0:1789760707.411801:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.411814:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452e5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.411823:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.411830:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760707.411835:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.411844:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452e5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.411863:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452e5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.411872:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452e5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760707.411956:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495728768:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48920us (49825us total) trans 4294972404 rc 0/0 00000100:00100000:3.0:1789760707.411966:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5116 00000100:00100000:3.0:1789760707.411975:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691634044947 time 691683738700 state 0x2 ref 0 00000100:00100000:0.0:1789760707.416253:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760707.416507:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495728896 00000100:00100000:1.0:1789760707.416516:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495728896 00000100:00100000:1.0:1789760707.416567:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691634044947 time 691683738700 state 0x0 ref 1 00000100:00100000:1.0:1789760707.416582:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691683999116 time 691683738700 state 0x1 ref 1 00000100:00100000:1.0:1789760707.416613:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691683999116 time 691737330071 state 0x0 ref 1 00000100:00100000:1.0:1789760707.416625:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495728896:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760707.416683:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20840448, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xa007 00010000:00010000:1.0:1789760707.416692:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545452ec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.416708:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:1.0:1789760707.416712:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760707.416718:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545452ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.416738:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545452ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.427968:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760707.515811:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20840448 -> 20844544 00010000:00010000:1.0:1789760707.515831:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545452ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.515843:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545452ec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.515854:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545452ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.515861:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760707.515866:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545452ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.515875:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545452ec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.515892:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545452ec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.515900:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545452ec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760707.515993:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495728896:12345-192.168.203.9@tcp:4:dd.0 Request processed in 99365us (99743us total) trans 4294972405 rc 0/0 00000100:00100000:1.0:1789760707.516003:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5117 00000100:00100000:1.0:1789760707.516013:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691683999116 time 691737330071 state 0x2 ref 0 00000001:00080000:0.0:1789760707.519324:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972405 is committed 00000100:00100000:1.0:1789760707.519536:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760707.519724:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495729024 00000100:00100000:3.0:1789760707.519733:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495729024 00000100:00100000:3.0:1789760707.519780:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691683999116 time 691737330071 state 0x0 ref 1 00000100:00100000:3.0:1789760707.519794:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691737590487 time 691737330071 state 0x1 ref 1 00000100:00100000:3.0:1789760707.519814:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691737590487 time 691840531277 state 0x0 ref 1 00000100:00100000:3.0:1789760707.519825:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495729024:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760707.519869:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20844544, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xa007 00010000:00010000:3.0:1789760707.519878:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452f3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.519892:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760707.519897:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760707.519903:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.519916:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760707.548913:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20844544 -> 20848640 00010000:00010000:1.0:1789760707.548934:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.548947:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452f3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.548957:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.548964:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760707.548968:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.548977:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452f3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.548996:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452f3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.549009:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545452f3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760707.549099:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495729024:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29270us (29563us total) trans 4294972406 rc 0/0 00000100:00100000:1.0:1789760707.549109:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5118 00000100:00100000:1.0:1789760707.549118:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691737590487 time 691840531277 state 0x2 ref 0 00000001:00080000:2.0:1789760707.550458:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972406 is committed 00000100:00100000:2.0:1789760707.553520:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760707.553604:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495729152 00000100:00100000:3.0:1789760707.553621:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495729152 00000100:00100000:3.0:1789760707.553671:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691737590487 time 691840531277 state 0x0 ref 1 00000100:00100000:3.0:1789760707.553686:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691840791693 time 691840531277 state 0x1 ref 1 00000100:00100000:3.0:1789760707.553715:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691840791693 time 691874432886 state 0x0 ref 1 00000100:00100000:3.0:1789760707.553727:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495729152:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760707.553769:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20848640, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xa007 00010000:00010000:3.0:1789760707.553777:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452fa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.553792:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760707.553796:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760707.553802:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.553815:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760707.584588:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20848640 -> 20852736 00010000:00010000:1.0:1789760707.584606:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.584629:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452fa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.584639:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.584647:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760707.584652:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.584661:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452fa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.584687:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452fa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.584695:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545452fa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760707.584782:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495729152:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31052us (31266us total) trans 4294972407 rc 0/0 00000100:00100000:1.0:1789760707.584793:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5119 00000100:00100000:1.0:1789760707.584803:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691840791693 time 691874432886 state 0x2 ref 0 00000001:00080000:3.0:1789760707.587882:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972407 is committed 00000100:00100000:0.0:1789760707.588134:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760707.588269:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495729280 00000100:00100000:1.0:1789760707.588398:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495729280 00000100:00100000:1.0:1789760707.588446:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691840791693 time 691874432886 state 0x0 ref 1 00000100:00100000:1.0:1789760707.588460:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691874693302 time 691874432886 state 0x1 ref 1 00000100:00100000:1.0:1789760707.588517:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691874693302 time 691909233315 state 0x0 ref 1 00000100:00100000:1.0:1789760707.588529:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495729280:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760707.588575:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20852736, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xa007 00010000:00010000:1.0:1789760707.588583:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545301 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.588599:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:1.0:1789760707.588603:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760707.588610:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545301 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.588633:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545301 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760707.618433:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20852736 -> 20856832 00010000:00010000:2.0:1789760707.618451:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545301 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.618465:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545301 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.618475:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545301 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.618482:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760707.618487:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545301 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.618496:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545301 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.618515:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545301 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.618522:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545301 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760707.618611:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495729280:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30079us (30478us total) trans 4294972408 rc 0/0 00000100:00100000:2.0:1789760707.618630:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5120 00000100:00100000:2.0:1789760707.618641:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691874693302 time 691909233315 state 0x2 ref 0 00000001:00080000:3.0:1789760707.618764:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972408 is committed 00000100:00100000:2.0:1789760707.622459:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760707.622978:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495729408 00000100:00100000:1.0:1789760707.622987:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495729408 00000100:00100000:1.0:1789760707.623040:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691874693302 time 691909233315 state 0x0 ref 1 00000100:00100000:1.0:1789760707.623056:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691909493731 time 691909233315 state 0x1 ref 1 00000100:00100000:1.0:1789760707.623085:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691909493731 time 691943802303 state 0x0 ref 1 00000100:00100000:1.0:1789760707.623097:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495729408:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760707.623150:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20856832, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xa007 00010000:00010000:1.0:1789760707.623159:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545308 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.623175:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:1.0:1789760707.623179:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760707.623185:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545308 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.623199:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545308 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760707.652875:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972409 is committed 00002000:00010000:3.0:1789760707.653153:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20856832 -> 20860928 00002000:00010000:3.0:1789760707.653158:0:6714:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760706 -> 1789760707 00002000:00010000:3.0:1789760707.653161:0:6714:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760706 -> 1789760707 00010000:00010000:3.0:1789760707.653174:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545308 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.653186:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545308 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.653200:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545308 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.653207:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760707.653212:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545308 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.653220:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545308 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.653238:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545308 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.653246:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545308 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760707.653488:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495729408:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30387us (31211us total) trans 4294972409 rc 0/0 00000100:00100000:3.0:1789760707.653499:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5121 00000100:00100000:3.0:1789760707.653509:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691909493731 time 691943802303 state 0x2 ref 0 00000100:00100000:1.0:1789760707.659342:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760707.659511:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495729536 00000100:00100000:3.0:1789760707.659520:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495729536 00000100:00100000:3.0:1789760707.659569:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691909493731 time 691943802303 state 0x0 ref 1 00000100:00100000:3.0:1789760707.659584:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691944062719 time 691943802303 state 0x1 ref 1 00000100:00100000:3.0:1789760707.659619:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691944062719 time 691980329392 state 0x0 ref 1 00000100:00100000:3.0:1789760707.659630:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495729536:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760707.659675:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20860928, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa007 00010000:00010000:3.0:1789760707.659683:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454530f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.659700:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760707.659704:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760707.659710:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454530f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.659724:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454530f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760707.686758:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972410 is committed 00002000:00010000:1.0:1789760707.687760:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20860928 -> 20865024 00010000:00010000:1.0:1789760707.687779:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454530f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.687792:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454530f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.687802:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454530f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.687810:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760707.687815:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454530f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.687824:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454530f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.687847:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454530f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.687855:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454530f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760707.687953:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495729536:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28319us (28615us total) trans 4294972410 rc 0/0 00000100:00100000:1.0:1789760707.687963:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5122 00000100:00100000:1.0:1789760707.687973:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691944062719 time 691980329392 state 0x2 ref 0 00000100:00100000:2.0:1789760707.698159:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760707.698266:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495729664 00000100:00100000:1.0:1789760707.698324:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495729664 00000100:00100000:0.0:1789760707.698549:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691944062719 time 691980329392 state 0x0 ref 1 00000100:00100000:0.0:1789760707.698565:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691980589808 time 691980329392 state 0x1 ref 1 00000100:00100000:0.0:1789760707.698598:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691980589808 time 692019315489 state 0x0 ref 1 00000100:00100000:0.0:1789760707.698624:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495729664:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760707.698673:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20865024, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa007 00010000:00010000:0.0:1789760707.698683:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545316 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.698700:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:0.0:1789760707.698704:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760707.698711:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545316 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.698726:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545316 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760707.723218:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20865024 -> 20869120 00010000:00010000:0.0:1789760707.723236:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545316 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.723248:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545316 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.723256:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545316 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.723264:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760707.723268:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545316 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.723311:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545316 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.723329:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545316 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.723336:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545316 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760707.723462:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972411 is committed 00000100:00100000:0.0:1789760707.723553:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495729664:12345-192.168.203.9@tcp:4:dd.0 Request processed in 24939us (25397us total) trans 4294972411 rc 0/0 00000100:00100000:0.0:1789760707.723563:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5123 00000100:00100000:0.0:1789760707.723573:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691980589808 time 692019315489 state 0x2 ref 0 00000100:00100000:0.0:1789760707.726533:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760707.726607:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495729792 00000100:00100000:3.0:1789760707.726673:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495729792 00000100:00100000:3.0:1789760707.726722:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 691980589808 time 692019315489 state 0x0 ref 1 00000100:00100000:3.0:1789760707.726736:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692019575905 time 692019315489 state 0x1 ref 1 00000100:00100000:3.0:1789760707.726787:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692019575905 time 692047503616 state 0x0 ref 1 00000100:00100000:3.0:1789760707.726799:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495729792:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760707.726841:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20869120, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa007 00010000:00010000:3.0:1789760707.726850:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454531d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.726866:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760707.726870:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760707.726876:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454531d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.726890:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454531d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760707.753317:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20869120 -> 20873216 00010000:00010000:1.0:1789760707.753336:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454531d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.753350:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454531d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.753360:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454531d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.753367:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760707.753372:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454531d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.753382:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454531d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.753400:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454531d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.753408:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454531d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760707.753488:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495729792:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26686us (26958us total) trans 4294972412 rc 0/0 00000100:00100000:1.0:1789760707.753498:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5124 00000100:00100000:1.0:1789760707.753507:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692019575905 time 692047503616 state 0x2 ref 0 00000100:00100000:1.0:1789760707.756761:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760707.756800:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495729920 00000100:00100000:1.0:1789760707.756809:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495729920 00000100:00100000:1.0:1789760707.756854:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692019575905 time 692047503616 state 0x0 ref 1 00000100:00100000:1.0:1789760707.756868:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692047764032 time 692047503616 state 0x1 ref 1 00000100:00100000:1.0:1789760707.756890:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692047764032 time 692077608115 state 0x0 ref 1 00000100:00100000:1.0:1789760707.756901:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495729920:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760707.756946:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20873216, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa007 00010000:00010000:1.0:1789760707.756954:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545324 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.756969:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760707.756973:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760707.756979:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545324 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.756992:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545324 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760707.757580:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972412 is committed 00000001:00080000:3.0:1789760707.786200:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972413 is committed 00002000:00010000:1.0:1789760707.786332:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20873216 -> 20877312 00010000:00010000:1.0:1789760707.786346:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545324 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.786360:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545324 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.786369:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545324 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.786376:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760707.786381:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545324 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.786390:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545324 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.786410:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545324 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.786418:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545324 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760707.786493:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495729920:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29589us (29735us total) trans 4294972413 rc 0/0 00000100:00100000:1.0:1789760707.786506:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5125 00000100:00100000:1.0:1789760707.786515:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692047764032 time 692077608115 state 0x2 ref 0 00000100:00100000:3.0:1789760707.789353:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760707.789446:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495730048 00000100:00100000:1.0:1789760707.789455:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495730048 00000100:00100000:1.0:1789760707.789506:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692047764032 time 692077608115 state 0x0 ref 1 00000100:00100000:1.0:1789760707.789521:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692077868531 time 692077608115 state 0x1 ref 1 00000100:00100000:1.0:1789760707.789542:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692077868531 time 692110259217 state 0x0 ref 1 00000100:00100000:1.0:1789760707.789553:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495730048:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760707.789611:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20877312, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa007 00010000:00010000:1.0:1789760707.789626:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454532b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.789641:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760707.789646:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760707.789652:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454532b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.789665:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454532b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760707.819512:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972414 is committed 00002000:00010000:1.0:1789760707.819597:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20877312 -> 20881408 00010000:00010000:1.0:1789760707.819612:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454532b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.819691:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454532b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.819700:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454532b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.819708:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760707.819713:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454532b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.819722:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454532b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.819741:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454532b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.819749:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454532b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760707.819834:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495730048:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30277us (30485us total) trans 4294972414 rc 0/0 00000100:00100000:1.0:1789760707.819843:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5126 00000100:00100000:1.0:1789760707.819852:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692077868531 time 692110259217 state 0x2 ref 0 00000100:00100000:0.0:1789760707.823782:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760707.823868:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495730176 00000100:00100000:1.0:1789760707.823878:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495730176 00000100:00100000:1.0:1789760707.823925:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692077868531 time 692110259217 state 0x0 ref 1 00000100:00100000:1.0:1789760707.823940:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692110519633 time 692110259217 state 0x1 ref 1 00000100:00100000:1.0:1789760707.823970:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692110519633 time 692144686317 state 0x0 ref 1 00000100:00100000:1.0:1789760707.823981:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495730176:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760707.824024:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20881408, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa007 00010000:00010000:1.0:1789760707.824032:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545332 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.824048:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760707.824052:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760707.824058:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545332 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.824071:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545332 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760707.850206:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20881408 -> 20885504 00010000:00010000:3.0:1789760707.850224:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545332 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.850238:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545332 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.850248:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545332 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.850256:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760707.850262:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545332 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.850307:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545332 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.850328:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545332 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.850335:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545332 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760707.850411:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495730176:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26427us (26634us total) trans 4294972415 rc 0/0 00000100:00100000:3.0:1789760707.850421:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5127 00000100:00100000:3.0:1789760707.850431:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692110519633 time 692144686317 state 0x2 ref 0 00000001:00080000:3.0:1789760707.853465:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972415 is committed 00000100:00100000:3.0:1789760707.853755:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760707.853842:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495730304 00000100:00100000:1.0:1789760707.853851:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495730304 00000100:00100000:1.0:1789760707.853902:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692110519633 time 692144686317 state 0x0 ref 1 00000100:00100000:1.0:1789760707.853917:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692144946733 time 692144686317 state 0x1 ref 1 00000100:00100000:1.0:1789760707.853945:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692144946733 time 692174662399 state 0x0 ref 1 00000100:00100000:1.0:1789760707.853960:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495730304:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760707.854004:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20885504, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa007 00010000:00010000:1.0:1789760707.854012:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354545339 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.854028:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760707.854032:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760707.854038:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354545339 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.854051:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354545339 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760707.879934:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972416 is committed 00002000:00010000:0.0:1789760707.880079:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20885504 -> 20889600 00010000:00010000:0.0:1789760707.880094:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354545339 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.880112:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354545339 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.880120:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354545339 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.880127:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760707.880132:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354545339 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.880140:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354545339 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.880160:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354545339 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.880168:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354545339 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760707.880262:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495730304:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26299us (26508us total) trans 4294972416 rc 0/0 00000100:00100000:0.0:1789760707.880320:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5128 00000100:00100000:0.0:1789760707.880331:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692144946733 time 692174662399 state 0x2 ref 0 00000100:00100000:3.0:1789760707.883957:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760707.884094:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495730432 00000100:00100000:1.0:1789760707.884105:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495730432 00000100:00100000:1.0:1789760707.884154:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692144946733 time 692174662399 state 0x0 ref 1 00000100:00100000:1.0:1789760707.884169:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692174922815 time 692174662399 state 0x1 ref 1 00000100:00100000:1.0:1789760707.884243:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692174922815 time 692204959942 state 0x0 ref 1 00000100:00100000:1.0:1789760707.884255:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495730432:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760707.884335:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20889600, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa007 00010000:00010000:1.0:1789760707.884344:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545340 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.884364:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760707.884368:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760707.884375:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545340 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.884388:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545340 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760707.906975:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972417 is committed 00002000:00010000:0.0:1789760707.907074:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20889600 -> 20893696 00010000:00010000:0.0:1789760707.907089:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545340 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.907102:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545340 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.907112:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545340 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.907119:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760707.907124:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545340 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.907133:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545340 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.907152:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545340 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.907160:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545340 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760707.907245:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495730432:12345-192.168.203.9@tcp:4:dd.0 Request processed in 22986us (23292us total) trans 4294972417 rc 0/0 00000100:00100000:0.0:1789760707.907255:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5129 00000100:00100000:0.0:1789760707.907265:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692174922815 time 692204959942 state 0x2 ref 0 00000100:00100000:0.0:1789760707.910201:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760707.910274:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495730560 00000100:00100000:3.0:1789760707.910323:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495730560 00000100:00100000:3.0:1789760707.910371:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692174922815 time 692204959942 state 0x0 ref 1 00000100:00100000:3.0:1789760707.910386:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692205220358 time 692204959942 state 0x1 ref 1 00000100:00100000:3.0:1789760707.910409:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692205220358 time 692231125670 state 0x0 ref 1 00000100:00100000:3.0:1789760707.910420:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495730560:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760707.910463:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20893696, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa007 00010000:00010000:3.0:1789760707.910472:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545347 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.910486:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760707.910491:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760707.910497:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545347 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.910510:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545347 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760707.942226:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972418 is committed 00002000:00010000:3.0:1789760707.942482:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20893696 -> 20897792 00010000:00010000:3.0:1789760707.942498:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545347 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.942512:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545347 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.942521:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545347 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.942529:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760707.942534:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545347 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.942544:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545347 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.942569:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545347 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.942577:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545347 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760707.942760:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495730560:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32337us (32562us total) trans 4294972418 rc 0/0 00000100:00100000:3.0:1789760707.942771:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5130 00000100:00100000:3.0:1789760707.942781:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692205220358 time 692231125670 state 0x2 ref 0 00000100:00100000:3.0:1789760707.946564:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760707.946783:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495730688 00000100:00100000:0.0:1789760707.946794:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495730688 00000100:00100000:0.0:1789760707.946842:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692205220358 time 692231125670 state 0x0 ref 1 00000100:00100000:0.0:1789760707.946857:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692231386086 time 692231125670 state 0x1 ref 1 00000100:00100000:0.0:1789760707.946927:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692231386086 time 692267643935 state 0x0 ref 1 00000100:00100000:0.0:1789760707.946940:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495730688:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760707.946985:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20897792, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa007 00010000:00010000:0.0:1789760707.946993:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454534e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.947008:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:0.0:1789760707.947012:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760707.947018:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454534e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.947031:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454534e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760707.972900:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20897792 -> 20901888 00010000:00010000:1.0:1789760707.972918:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454534e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.972930:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454534e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.972939:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454534e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.972947:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760707.972951:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454534e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.972959:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454534e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.972983:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454534e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.972989:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454534e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760707.973071:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495730688:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26128us (26510us total) trans 4294972419 rc 0/0 00000100:00100000:1.0:1789760707.973081:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5131 00000100:00100000:1.0:1789760707.973091:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692231386086 time 692267643935 state 0x2 ref 0 00000001:00080000:3.0:1789760707.974255:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972419 is committed 00000100:00100000:0.0:1789760707.976320:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760707.976440:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495730816 00000100:00100000:1.0:1789760707.976449:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495730816 00000100:00100000:1.0:1789760707.976497:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692231386086 time 692267643935 state 0x0 ref 1 00000100:00100000:1.0:1789760707.976511:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692267904351 time 692267643935 state 0x1 ref 1 00000100:00100000:1.0:1789760707.976529:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692267904351 time 692297246422 state 0x0 ref 1 00000100:00100000:1.0:1789760707.976540:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495730816:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760707.976583:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20901888, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa007 00010000:00010000:1.0:1789760707.976592:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545355 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.976608:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760707.976622:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760707.976628:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545355 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.976642:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545355 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760708.012322:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972420 is committed 00002000:00010000:3.0:1789760708.012384:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20901888 -> 20905984 00010000:00010000:3.0:1789760708.012398:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545355 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.012415:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545355 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.012425:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545355 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.012433:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760708.012437:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545355 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.012447:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545355 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.012465:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545355 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.012473:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545355 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760708.012557:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495730816:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36012us (36239us total) trans 4294972420 rc 0/0 00000100:00100000:3.0:1789760708.012567:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5132 00000100:00100000:3.0:1789760708.012577:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692267904351 time 692297246422 state 0x2 ref 0 00000100:00100000:0.0:1789760708.017535:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760708.017739:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495730944 00000100:00100000:3.0:1789760708.017748:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495730944 00000100:00100000:3.0:1789760708.017804:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692267904351 time 692297246422 state 0x0 ref 1 00000100:00100000:3.0:1789760708.017819:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692297506838 time 692297246422 state 0x1 ref 1 00000100:00100000:3.0:1789760708.018187:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692297506838 time 692338902598 state 0x0 ref 1 00000100:00100000:3.0:1789760708.018202:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495730944:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760708.018245:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20905984, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa007 00010000:00010000:3.0:1789760708.018254:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454535c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.018269:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760708.018274:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760708.018308:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454535c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.018322:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454535c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760708.056046:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972421 is committed 00002000:00010000:3.0:1789760708.056165:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20905984 -> 20910080 00010000:00010000:3.0:1789760708.056179:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454535c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.056193:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454535c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.056202:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454535c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.056210:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760708.056215:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454535c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.056225:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454535c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.056242:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454535c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.056250:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454535c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760708.056416:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495730944:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38212us (38884us total) trans 4294972421 rc 0/0 00000100:00100000:3.0:1789760708.056426:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5133 00000100:00100000:3.0:1789760708.056435:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692297506838 time 692338902598 state 0x2 ref 0 00000100:00100000:1.0:1789760708.060184:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760708.060589:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495731072 00000100:00100000:0.0:1789760708.060598:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495731072 00000100:00100000:0.0:1789760708.061796:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692297506838 time 692338902598 state 0x0 ref 1 00000100:00100000:0.0:1789760708.061813:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692339163014 time 692338902598 state 0x1 ref 1 00000100:00100000:0.0:1789760708.061847:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692339163014 time 692382564550 state 0x0 ref 1 00000100:00100000:0.0:1789760708.061860:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495731072:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760708.061903:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20910080, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa007 00010000:00010000:0.0:1789760708.061911:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545363 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.061926:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:0.0:1789760708.061931:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760708.061938:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545363 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.061952:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545363 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760708.084236:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760708.084314:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760708.084374:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760708.084401:0:8003:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495731328 00000100:00100000:3.0:1789760708.084410:0:8003:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495731328 00000100:00100000:3.0:1789760708.084454:0:8003:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495731456 00000100:00100000:3.0:1789760708.084456:0:8003:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495731456 00000100:00100000:3.0:1789760708.084475:0:8003:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 450 00000100:00100000:3.0:1789760708.084494:0:8003:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:17022965-1e4f-46fb-915c-61730096b8e2+8:2356:x1876699495731328:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:3.0:1789760708.084557:0:8003:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:17022965-1e4f-46fb-915c-61730096b8e2+8:2356:x1876699495731328:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 75us (330us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760708.084565:0:8003:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 450 00000100:00100000:3.0:1789760708.084580:0:8003:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 451 00000100:00100000:3.0:1789760708.084585:0:8003:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2356:x1876699495731456:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:3.0:1789760708.084605:0:8003:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2356:x1876699495731456:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 20us (231us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760708.084625:0:8003:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 451 00000100:00100000:3.0:1789760708.084642:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495731200 00000100:00100000:3.0:1789760708.084645:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495731200 00000100:00100000:3.0:1789760708.084666:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 388 00000100:00100000:3.0:1789760708.084671:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929012315c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2356:x1876699495731200:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:3.0:1789760708.084692:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929012315c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2356:x1876699495731200:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 22us (422us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760708.084698:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 388 00000100:00080000:3.0:1789760708.084722:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:1.0:1789760708.109542:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972422 is committed 00002000:00010000:3.0:1789760708.109724:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20910080 -> 20914176 00010000:00010000:3.0:1789760708.109738:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545363 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.109751:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545363 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.109761:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545363 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.109769:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760708.109774:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545363 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.109783:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545363 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.109807:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545363 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.109816:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545363 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760708.109942:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495731072:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48080us (49761us total) trans 4294972422 rc 0/0 00000100:00100000:3.0:1789760708.109952:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5134 00000100:00100000:3.0:1789760708.109962:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692339163014 time 692382564550 state 0x2 ref 0 00000100:00100000:3.0:1789760708.113887:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760708.113993:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495731584 00000100:00100000:1.0:1789760708.114003:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495731584 00000100:00100000:1.0:1789760708.114059:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692339163014 time 692382564550 state 0x0 ref 1 00000100:00100000:1.0:1789760708.114073:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692382824966 time 692382564550 state 0x1 ref 1 00000100:00100000:1.0:1789760708.114108:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692382824966 time 692434824240 state 0x0 ref 1 00000100:00100000:1.0:1789760708.114120:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495731584:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760708.114167:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20914176, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa007 00010000:00010000:1.0:1789760708.114177:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454536a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.114192:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760708.114196:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760708.114203:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454536a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.114221:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454536a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760708.230053:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972423 is committed 00002000:00010000:1.0:1789760708.230211:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20914176 -> 20918272 00010000:00010000:1.0:1789760708.230226:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454536a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.230240:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454536a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.230249:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454536a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.230257:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760708.230261:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454536a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.230317:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454536a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.230340:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454536a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.230347:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454536a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760708.230447:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495731584:12345-192.168.203.9@tcp:4:dd.0 Request processed in 116324us (116567us total) trans 4294972423 rc 0/0 00000100:00100000:1.0:1789760708.230457:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5135 00000100:00100000:1.0:1789760708.230467:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692382824966 time 692434824240 state 0x2 ref 0 00000100:00100000:1.0:1789760708.234759:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760708.234832:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495731712 00000100:00100000:3.0:1789760708.234842:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495731712 00000100:00100000:3.0:1789760708.234894:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692382824966 time 692434824240 state 0x0 ref 1 00000100:00100000:3.0:1789760708.234908:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692435084656 time 692434824240 state 0x1 ref 1 00000100:00100000:3.0:1789760708.234929:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692435084656 time 692555646492 state 0x0 ref 1 00000100:00100000:3.0:1789760708.234940:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92902dbc4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495731712:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760708.234987:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20918272, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa007 00010000:00010000:3.0:1789760708.234996:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545371 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.235017:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760708.235021:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760708.235027:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545371 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.235041:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545371 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760708.275333:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972424 is committed 00002000:00010000:3.0:1789760708.275490:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20918272 -> 20922368 00010000:00010000:3.0:1789760708.275504:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545371 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.275517:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545371 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.275526:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545371 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.275534:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760708.275539:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545371 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.275548:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545371 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.275568:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545371 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.275577:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545371 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760708.275670:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92902dbc4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495731712:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40727us (40915us total) trans 4294972424 rc 0/0 00000100:00100000:3.0:1789760708.275680:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5136 00000100:00100000:3.0:1789760708.275689:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692435084656 time 692555646492 state 0x2 ref 0 00000100:00100000:2.0:1789760708.279403:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760708.279499:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495731840 00000100:00100000:3.0:1789760708.279509:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495731840 00000100:00100000:3.0:1789760708.279561:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692435084656 time 692555646492 state 0x0 ref 1 00000100:00100000:3.0:1789760708.279575:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692555906908 time 692555646492 state 0x1 ref 1 00000100:00100000:3.0:1789760708.279603:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692555906908 time 692600320327 state 0x0 ref 1 00000100:00100000:3.0:1789760708.279622:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495731840:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760708.279669:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20922368, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa007 00010000:00010000:3.0:1789760708.279678:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545378 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.279696:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760708.279700:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760708.279706:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545378 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.279726:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545378 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760708.325579:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972425 is committed 00002000:00010000:3.0:1789760708.325822:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20922368 -> 20926464 00010000:00010000:3.0:1789760708.325839:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545378 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.325853:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545378 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.325862:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545378 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.325869:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760708.325874:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545378 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.325882:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545378 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.325901:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545378 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.325909:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545378 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760708.326005:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495731840:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46379us (46606us total) trans 4294972425 rc 0/0 00000100:00100000:3.0:1789760708.326015:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5137 00000100:00100000:3.0:1789760708.326026:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692555906908 time 692600320327 state 0x2 ref 0 00000100:00100000:0.0:1789760708.330812:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760708.330890:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495731968 00000100:00100000:1.0:1789760708.330900:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495731968 00000100:00100000:1.0:1789760708.330955:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692555906908 time 692600320327 state 0x0 ref 1 00000100:00100000:1.0:1789760708.330970:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692600580743 time 692600320327 state 0x1 ref 1 00000100:00100000:1.0:1789760708.330998:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692600580743 time 692651715278 state 0x0 ref 1 00000100:00100000:1.0:1789760708.331010:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495731968:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760708.331061:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20926464, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa007 00010000:00010000:1.0:1789760708.331070:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454537f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.331085:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760708.331090:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760708.331096:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454537f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.331110:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454537f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760708.382402:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972426 is committed 00002000:00010000:1.0:1789760708.382543:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20926464 -> 20930560 00010000:00010000:1.0:1789760708.382558:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454537f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.382570:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454537f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.382578:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454537f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.382585:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760708.382589:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454537f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.382597:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454537f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.382617:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454537f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.382627:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454537f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760708.382943:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495731968:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51928us (52134us total) trans 4294972426 rc 0/0 00000100:00100000:1.0:1789760708.382958:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5138 00000100:00100000:1.0:1789760708.382970:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692600580743 time 692651715278 state 0x2 ref 0 00000100:00100000:2.0:1789760708.390882:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760708.395948:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495732096 00000100:00100000:3.0:1789760708.395964:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495732096 00000100:00100000:3.0:1789760708.396029:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692600580743 time 692651715278 state 0x0 ref 1 00000100:00100000:3.0:1789760708.396045:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692651975694 time 692651715278 state 0x1 ref 1 00000100:00100000:3.0:1789760708.396078:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692651975694 time 692716795383 state 0x0 ref 1 00000100:00100000:3.0:1789760708.396091:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495732096:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760708.396142:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20930560, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa007 00010000:00010000:3.0:1789760708.396151:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545386 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.396167:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760708.396172:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760708.396178:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545386 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.396193:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545386 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.446996:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760708.473479:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972427 is committed 00002000:00010000:1.0:1789760708.475585:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20930560 -> 20934656 00010000:00010000:1.0:1789760708.475605:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545386 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.475619:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545386 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.475629:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545386 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.475636:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760708.475642:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545386 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.475660:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545386 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.475682:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545386 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.475692:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545386 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760708.475795:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495732096:12345-192.168.203.9@tcp:4:dd.0 Request processed in 79701us (84917us total) trans 4294972427 rc 0/0 00000100:00100000:1.0:1789760708.475807:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5139 00000100:00100000:1.0:1789760708.475818:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692651975694 time 692716795383 state 0x2 ref 0 00000100:00100000:3.0:1789760708.509908:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760708.510452:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495732224 00000100:00100000:0.0:1789760708.510462:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495732224 00000100:00100000:0.0:1789760708.510518:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692651975694 time 692716795383 state 0x0 ref 1 00000100:00100000:0.0:1789760708.510533:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692717055799 time 692716795383 state 0x1 ref 1 00000100:00100000:0.0:1789760708.510567:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692717055799 time 692831284340 state 0x0 ref 1 00000100:00100000:0.0:1789760708.510580:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495732224:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760708.510628:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20934656, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa007 00010000:00010000:0.0:1789760708.510637:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454538d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.510654:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:0.0:1789760708.510658:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760708.510665:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454538d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.510678:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454538d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760708.573837:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972428 is committed 00002000:00010000:0.0:1789760708.573922:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20934656 -> 20938752 00010000:00010000:0.0:1789760708.573938:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454538d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.573950:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454538d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.573960:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454538d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.573966:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760708.573972:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454538d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.573982:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454538d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.574001:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454538d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.574008:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454538d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760708.574093:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495732224:12345-192.168.203.9@tcp:4:dd.0 Request processed in 63511us (64190us total) trans 4294972428 rc 0/0 00000100:00100000:0.0:1789760708.574103:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5140 00000100:00100000:0.0:1789760708.574113:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692717055799 time 692831284340 state 0x2 ref 0 00000100:00100000:1.0:1789760708.587508:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760708.588218:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495732352 00000100:00100000:2.0:1789760708.588229:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495732352 00000100:00100000:2.0:1789760708.588335:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692717055799 time 692831284340 state 0x0 ref 1 00000100:00100000:2.0:1789760708.588350:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692831544756 time 692831284340 state 0x1 ref 1 00000100:00100000:2.0:1789760708.588378:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692831544756 time 692909095033 state 0x0 ref 1 00000100:00100000:2.0:1789760708.588390:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495732352:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760708.588434:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20938752, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa007 00010000:00010000:2.0:1789760708.588443:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545394 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.588466:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:2.0:1789760708.588470:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760708.588476:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545394 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.588490:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545394 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760708.643145:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20938752 -> 20942848 00010000:00010000:2.0:1789760708.643163:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545394 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.643177:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545394 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.643187:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545394 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.643194:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760708.643199:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545394 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.643208:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545394 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.643227:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545394 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.643235:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545394 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760708.643366:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495732352:12345-192.168.203.9@tcp:4:dd.0 Request processed in 54973us (55862us total) trans 4294972429 rc 0/0 00000100:00100000:2.0:1789760708.643376:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5141 00000100:00100000:2.0:1789760708.643385:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692831544756 time 692909095033 state 0x2 ref 0 00000001:00080000:3.0:1789760708.643551:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972429 is committed 00000100:00100000:2.0:1789760708.646853:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760708.646910:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495732480 00000100:00100000:1.0:1789760708.646920:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495732480 00000100:00100000:1.0:1789760708.646969:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692831544756 time 692909095033 state 0x0 ref 1 00000100:00100000:1.0:1789760708.646984:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692909355449 time 692909095033 state 0x1 ref 1 00000100:00100000:1.0:1789760708.647010:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692909355449 time 692967727136 state 0x0 ref 1 00000100:00100000:1.0:1789760708.647022:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495732480:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760708.647067:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20942848, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa007 00010000:00010000:1.0:1789760708.647077:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454539b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.647093:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760708.647098:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760708.647105:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454539b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.647123:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454539b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760708.704596:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20942848 -> 20946944 00002000:00010000:0.0:1789760708.704607:0:6714:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760707 -> 1789760708 00002000:00010000:0.0:1789760708.704609:0:6714:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760707 -> 1789760708 00010000:00010000:0.0:1789760708.704622:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454539b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.704636:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454539b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.704645:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454539b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.704653:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760708.704658:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454539b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.704667:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454539b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.704698:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454539b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.704706:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454539b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760708.704815:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495732480:12345-192.168.203.9@tcp:4:dd.0 Request processed in 57789us (57967us total) trans 4294972430 rc 0/0 00000100:00100000:0.0:1789760708.704825:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5142 00000100:00100000:0.0:1789760708.704836:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692909355449 time 692967727136 state 0x2 ref 0 00000001:00080000:3.0:1789760708.705017:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972430 is committed 00000100:00100000:3.0:1789760708.708421:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760708.709073:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495732608 00000100:00100000:0.0:1789760708.709084:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495732608 00000100:00100000:0.0:1789760708.709136:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692909355449 time 692967727136 state 0x0 ref 1 00000100:00100000:0.0:1789760708.709152:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692967987552 time 692967727136 state 0x1 ref 1 00000100:00100000:0.0:1789760708.709182:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692967987552 time 693029899190 state 0x0 ref 1 00000100:00100000:0.0:1789760708.709199:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495732608:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760708.709244:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20946944, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xa007 00010000:00010000:0.0:1789760708.709252:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545453a2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.709267:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:0.0:1789760708.709271:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760708.709277:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545453a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.709327:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545453a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760708.764597:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972431 is committed 00002000:00010000:1.0:1789760708.764807:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20946944 -> 20951040 00010000:00010000:1.0:1789760708.764822:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545453a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.764835:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545453a2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.764845:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545453a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.764853:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760708.764858:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545453a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.764868:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545453a2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.764885:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545453a2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.764893:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545453a2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760708.765019:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495732608:12345-192.168.203.9@tcp:4:dd.0 Request processed in 55818us (56602us total) trans 4294972431 rc 0/0 00000100:00100000:1.0:1789760708.765029:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5143 00000100:00100000:1.0:1789760708.765039:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692967987552 time 693029899190 state 0x2 ref 0 00000100:00100000:1.0:1789760708.773930:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760708.774022:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495732736 00000100:00100000:2.0:1789760708.774032:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495732736 00000100:00100000:2.0:1789760708.774087:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 692967987552 time 693029899190 state 0x0 ref 1 00000100:00100000:2.0:1789760708.774103:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693030159606 time 693029899190 state 0x1 ref 1 00000100:00100000:2.0:1789760708.774131:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693030159606 time 693094848807 state 0x0 ref 1 00000100:00100000:2.0:1789760708.774143:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495732736:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760708.774192:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20951040, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xa007 00010000:00010000:2.0:1789760708.774201:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545453a9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.774217:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:2.0:1789760708.774222:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760708.774228:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545453a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.774243:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545453a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760708.853191:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20951040 -> 20955136 00010000:00010000:2.0:1789760708.853211:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545453a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.853224:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545453a9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.853234:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545453a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.853242:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760708.853248:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545453a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.853258:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545453a9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.853317:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545453a9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.853327:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545453a9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760708.853538:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495732736:12345-192.168.203.9@tcp:4:dd.0 Request processed in 79389us (79610us total) trans 4294972432 rc 0/0 00000100:00100000:2.0:1789760708.853550:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5144 00000100:00100000:2.0:1789760708.853561:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693030159606 time 693094848807 state 0x2 ref 0 00000001:00080000:3.0:1789760708.853655:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972432 is committed 00000100:00100000:2.0:1789760708.857191:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760708.857533:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495732864 00000100:00100000:0.0:1789760708.857544:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495732864 00000100:00100000:0.0:1789760708.857597:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693030159606 time 693094848807 state 0x0 ref 1 00000100:00100000:0.0:1789760708.857677:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693095109223 time 693094848807 state 0x1 ref 1 00000100:00100000:0.0:1789760708.857743:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693095109223 time 693178458697 state 0x0 ref 1 00000100:00100000:0.0:1789760708.857755:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495732864:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760708.857804:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20955136, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xa007 00010000:00010000:0.0:1789760708.857814:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545453b0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.857835:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:0.0:1789760708.857839:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760708.857845:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545453b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.857859:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545453b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760708.887080:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972433 is committed 00002000:00010000:2.0:1789760708.887192:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20955136 -> 20959232 00010000:00010000:2.0:1789760708.887207:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545453b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.887222:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545453b0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.887232:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545453b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.887238:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760708.887244:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545453b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.887253:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545453b0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.887317:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545453b0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.887332:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545453b0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760708.887428:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495732864:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29669us (30242us total) trans 4294972433 rc 0/0 00000100:00100000:2.0:1789760708.887439:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5145 00000100:00100000:2.0:1789760708.887450:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693095109223 time 693178458697 state 0x2 ref 0 00000100:00100000:0.0:1789760708.895953:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760708.896086:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495732992 00000100:00100000:3.0:1789760708.896095:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495732992 00000100:00100000:3.0:1789760708.896150:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693095109223 time 693178458697 state 0x0 ref 1 00000100:00100000:3.0:1789760708.896165:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693178719113 time 693178458697 state 0x1 ref 1 00000100:00100000:3.0:1789760708.896195:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693178719113 time 693216912580 state 0x0 ref 1 00000100:00100000:3.0:1789760708.896209:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f873480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495732992:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760708.896252:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20959232, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xa007 00010000:00010000:3.0:1789760708.896261:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545453b7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.896316:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760708.896321:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760708.896328:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545453b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.896341:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545453b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760708.930797:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972434 is committed 00002000:00010000:2.0:1789760708.930893:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20959232 -> 20963328 00010000:00010000:2.0:1789760708.930910:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545453b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.930924:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545453b7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.930933:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545453b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.930940:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760708.930945:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545453b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.930955:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545453b7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.930980:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545453b7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.930987:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545453b7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760708.931079:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f873480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495732992:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34868us (35132us total) trans 4294972434 rc 0/0 00000100:00100000:2.0:1789760708.931090:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5146 00000100:00100000:2.0:1789760708.931101:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693178719113 time 693216912580 state 0x2 ref 0 00000100:00100000:0.0:1789760708.935359:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760708.935551:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495733120 00000100:00100000:2.0:1789760708.935561:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495733120 00000100:00100000:2.0:1789760708.935627:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693178719113 time 693216912580 state 0x0 ref 1 00000100:00100000:2.0:1789760708.935641:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693217172996 time 693216912580 state 0x1 ref 1 00000100:00100000:2.0:1789760708.935669:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693217172996 time 693256386197 state 0x0 ref 1 00000100:00100000:2.0:1789760708.935681:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f871500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495733120:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760708.935726:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20963328, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xa007 00010000:00010000:2.0:1789760708.935735:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545453be lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.935752:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:2.0:1789760708.935756:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760708.935763:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545453be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.935775:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545453be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760708.992179:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972435 is committed 00002000:00010000:3.0:1789760708.992235:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20963328 -> 20967424 00010000:00010000:3.0:1789760708.992260:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545453be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.992273:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545453be lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.992322:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545453be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.992330:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760708.992335:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545453be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.992343:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545453be lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.992366:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545453be lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.992373:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545453be lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760708.992697:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f871500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495733120:12345-192.168.203.9@tcp:4:dd.0 Request processed in 57010us (57340us total) trans 4294972435 rc 0/0 00000100:00100000:2.0:1789760708.992708:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5147 00000100:00100000:2.0:1789760708.992718:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693217172996 time 693256386197 state 0x2 ref 0 00000100:00100000:1.0:1789760709.000562:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760709.000710:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495733248 00000100:00100000:2.0:1789760709.000719:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495733248 00000100:00100000:2.0:1789760709.000788:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693217172996 time 693256386197 state 0x0 ref 1 00000100:00100000:2.0:1789760709.000804:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693256646613 time 693256386197 state 0x1 ref 1 00000100:00100000:2.0:1789760709.000833:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693256646613 time 693321550579 state 0x0 ref 1 00000100:00100000:2.0:1789760709.000846:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495733248:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760709.000958:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20967424, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xa007 00010000:00010000:2.0:1789760709.000967:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545453c5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.000984:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:2.0:1789760709.000988:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760709.000996:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545453c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.001009:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545453c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760709.002480:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789760709.055262:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20967424 -> 20971520 00010000:00010000:2.0:1789760709.055325:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545453c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.055339:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545453c5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.055349:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545453c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.055357:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760709.055362:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545453c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.055372:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545453c5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.055391:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545453c5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.055398:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545453c5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760709.055484:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495733248:12345-192.168.203.9@tcp:4:dd.0 Request processed in 54635us (54924us total) trans 4294972436 rc 0/0 00000100:00100000:2.0:1789760709.055502:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5148 00000100:00100000:2.0:1789760709.055512:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693256646613 time 693321550579 state 0x2 ref 0 00000001:00080000:1.0:1789760709.055828:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972436 is committed 00000100:00100000:0.0:1789760709.061449:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760709.061579:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495733376 00000100:00100000:2.0:1789760709.061588:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495733376 00000100:00100000:2.0:1789760709.061636:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693256646613 time 693321550579 state 0x0 ref 1 00000100:00100000:2.0:1789760709.061653:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693321810995 time 693321550579 state 0x1 ref 1 00000100:00100000:2.0:1789760709.061674:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693321810995 time 693382390675 state 0x0 ref 1 00000100:00100000:2.0:1789760709.061685:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f871f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495733376:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760709.061730:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20971520, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xa007 00010000:00010000:2.0:1789760709.061739:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453cc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.061755:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:2.0:1789760709.061760:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760709.061766:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.061779:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760709.141369:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972437 is committed 00002000:00010000:2.0:1789760709.144893:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20971520 -> 20975616 00002000:00010000:2.0:1789760709.144903:0:6713:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 40967 -> 43015 00010000:00010000:2.0:1789760709.144916:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.144931:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453cc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.144942:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.144950:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760709.144955:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.144964:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453cc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.144986:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453cc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.144994:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453cc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760709.145158:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f871f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495733376:12345-192.168.203.9@tcp:4:dd.0 Request processed in 83469us (83711us total) trans 4294972437 rc 0/0 00000100:00100000:3.0:1789760709.145169:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5149 00000100:00100000:3.0:1789760709.145179:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693321810995 time 693382390675 state 0x2 ref 0 00000100:00100000:0.0:1789760709.151209:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760709.151657:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495733504 00000100:00100000:2.0:1789760709.151668:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495733504 00000100:00100000:3.0:1789760709.151889:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693321810995 time 693382390675 state 0x0 ref 1 00000100:00100000:3.0:1789760709.151908:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693382651091 time 693382390675 state 0x1 ref 1 00000100:00100000:3.0:1789760709.152080:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693382651091 time 693472794950 state 0x0 ref 1 00000100:00100000:3.0:1789760709.152093:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495733504:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760709.152141:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20975616, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xa807 00010000:00010000:3.0:1789760709.152151:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545453d3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.152169:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760709.152173:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760709.152180:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545453d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.152194:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545453d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760709.183922:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972438 is committed 00002000:00010000:2.0:1789760709.184021:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20975616 -> 20979712 00010000:00010000:2.0:1789760709.184039:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545453d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.184053:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545453d3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.184063:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545453d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.184071:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760709.184076:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545453d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.184085:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545453d3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.184102:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545453d3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.184109:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545453d3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760709.184206:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495733504:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32110us (33001us total) trans 4294972438 rc 0/0 00000100:00100000:2.0:1789760709.184216:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5150 00000100:00100000:2.0:1789760709.184226:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693382651091 time 693472794950 state 0x2 ref 0 00000100:00100000:1.0:1789760709.194194:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760709.194316:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495733632 00000100:00100000:2.0:1789760709.194328:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495733632 00000100:00100000:2.0:1789760709.194384:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693382651091 time 693472794950 state 0x0 ref 1 00000100:00100000:2.0:1789760709.194400:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693473055366 time 693472794950 state 0x1 ref 1 00000100:00100000:2.0:1789760709.194429:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693473055366 time 693515146188 state 0x0 ref 1 00000100:00100000:2.0:1789760709.194451:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495733632:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760709.194509:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20979712, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xa807 00010000:00010000:2.0:1789760709.194519:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453da lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.194536:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:2.0:1789760709.194541:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760709.194548:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.194563:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760709.248309:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972439 is committed 00002000:00010000:2.0:1789760709.248399:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20979712 -> 20983808 00010000:00010000:2.0:1789760709.248416:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.248433:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453da lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.248442:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.248450:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760709.248455:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.248464:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453da lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.248484:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453da lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.248493:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453da lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760709.248584:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495733632:12345-192.168.203.9@tcp:4:dd.0 Request processed in 54141us (54397us total) trans 4294972439 rc 0/0 00000100:00100000:2.0:1789760709.248595:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5151 00000100:00100000:2.0:1789760709.248606:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693473055366 time 693515146188 state 0x2 ref 0 00000100:00100000:0.0:1789760709.251875:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760709.251986:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495733760 00000100:00100000:3.0:1789760709.251996:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495733760 00000100:00100000:3.0:1789760709.252049:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693473055366 time 693515146188 state 0x0 ref 1 00000100:00100000:3.0:1789760709.252063:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693515406604 time 693515146188 state 0x1 ref 1 00000100:00100000:3.0:1789760709.252156:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693515406604 time 693572872377 state 0x0 ref 1 00000100:00100000:3.0:1789760709.252170:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495733760:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760709.252218:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20983808, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xa807 00010000:00010000:3.0:1789760709.252227:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545453e1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.252242:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760709.252247:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760709.252254:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545453e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.252355:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545453e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760709.318429:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972440 is committed 00002000:00010000:3.0:1789760709.318787:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20983808 -> 20987904 00010000:00010000:3.0:1789760709.318805:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545453e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.318819:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545453e1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.318834:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545453e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.318842:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760709.318847:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545453e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.318857:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545453e1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.318877:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545453e1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.318885:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545453e1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760709.318974:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495733760:12345-192.168.203.9@tcp:4:dd.0 Request processed in 66802us (67105us total) trans 4294972440 rc 0/0 00000100:00100000:3.0:1789760709.318986:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5152 00000100:00100000:3.0:1789760709.318997:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693515406604 time 693572872377 state 0x2 ref 0 00000100:00100000:0.0:1789760709.326727:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760709.326833:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495733888 00000100:00100000:2.0:1789760709.326842:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495733888 00000100:00100000:2.0:1789760709.326960:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693515406604 time 693572872377 state 0x0 ref 1 00000100:00100000:2.0:1789760709.326975:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693573132793 time 693572872377 state 0x1 ref 1 00000100:00100000:2.0:1789760709.327007:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693573132793 time 693647723459 state 0x0 ref 1 00000100:00100000:2.0:1789760709.327022:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495733888:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760709.327074:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20987904, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xa807 00010000:00010000:2.0:1789760709.327084:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453e8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.327100:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:2.0:1789760709.327105:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760709.327111:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.327126:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760709.389444:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20987904 -> 20992000 00010000:00010000:2.0:1789760709.389466:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.389479:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453e8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.389494:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.389502:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760709.389508:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.389518:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453e8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.389539:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453e8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.389547:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453e8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760709.389648:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495733888:12345-192.168.203.9@tcp:4:dd.0 Request processed in 62625us (62925us total) trans 4294972441 rc 0/0 00000100:00100000:2.0:1789760709.389658:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5153 00000100:00100000:2.0:1789760709.389668:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693573132793 time 693647723459 state 0x2 ref 0 00000001:00080000:3.0:1789760709.390126:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972441 is committed 00000100:00100000:1.0:1789760709.400394:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760709.400487:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495734016 00000100:00100000:2.0:1789760709.400498:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495734016 00000100:00100000:2.0:1789760709.400551:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693573132793 time 693647723459 state 0x0 ref 1 00000100:00100000:2.0:1789760709.400567:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693647983875 time 693647723459 state 0x1 ref 1 00000100:00100000:2.0:1789760709.400589:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693647983875 time 693721306366 state 0x0 ref 1 00000100:00100000:2.0:1789760709.400600:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495734016:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760709.400654:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20992000, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xa807 00010000:00010000:2.0:1789760709.400663:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453ef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.400679:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:2.0:1789760709.400684:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760709.400691:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.400706:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760709.462011:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972442 is committed 00002000:00010000:2.0:1789760709.468241:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20992000 -> 20996096 00010000:00010000:2.0:1789760709.468273:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.468317:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453ef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.468327:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.468334:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760709.468339:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.468349:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.468370:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453ef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.468377:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545453ef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760709.468479:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495734016:12345-192.168.203.9@tcp:4:dd.0 Request processed in 67876us (68091us total) trans 4294972442 rc 0/0 00000100:00100000:2.0:1789760709.468489:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5154 00000100:00100000:2.0:1789760709.468500:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693647983875 time 693721306366 state 0x2 ref 0 00010000:00010000:2.0:1789760709.471589:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760709.478644:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760709.478749:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495734144 00000100:00100000:3.0:1789760709.478762:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495734144 00000100:00100000:3.0:1789760709.478825:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693647983875 time 693721306366 state 0x0 ref 1 00000100:00100000:3.0:1789760709.478842:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693721566782 time 693721306366 state 0x1 ref 1 00000100:00100000:3.0:1789760709.478874:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693721566782 time 693799590870 state 0x0 ref 1 00000100:00100000:3.0:1789760709.478886:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903639c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495734144:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760709.478935:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20996096, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xa807 00010000:00010000:3.0:1789760709.478945:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545453f6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.478962:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760709.478967:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760709.478973:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545453f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.478989:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545453f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760709.556899:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972443 is committed 00002000:00010000:1.0:1789760709.557007:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20996096 -> 21000192 00010000:00010000:1.0:1789760709.557024:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545453f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.557037:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545453f6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.557045:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545453f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.557052:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760709.557056:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545453f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.557065:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545453f6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.557085:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545453f6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.557093:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545453f6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760709.557189:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903639c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495734144:12345-192.168.203.9@tcp:4:dd.0 Request processed in 78301us (78553us total) trans 4294972443 rc 0/0 00000100:00100000:1.0:1789760709.557200:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5155 00000100:00100000:1.0:1789760709.557210:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693721566782 time 693799590870 state 0x2 ref 0 00000100:00100000:1.0:1789760709.560637:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760709.560864:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495734272 00000100:00100000:2.0:1789760709.560873:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495734272 00000100:00100000:2.0:1789760709.560924:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693721566782 time 693799590870 state 0x0 ref 1 00000100:00100000:2.0:1789760709.560940:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693799851286 time 693799590870 state 0x1 ref 1 00000100:00100000:2.0:1789760709.560970:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693799851286 time 693881687027 state 0x0 ref 1 00000100:00100000:2.0:1789760709.560983:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495734272:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760709.561035:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21000192, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xa807 00010000:00010000:2.0:1789760709.561044:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545453fd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.561060:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:2.0:1789760709.561064:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760709.561070:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545453fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.561083:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545453fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760709.598531:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972444 is committed 00002000:00010000:0.0:1789760709.599726:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21000192 -> 21004288 00010000:00010000:0.0:1789760709.599746:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545453fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.599759:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545453fd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.599768:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545453fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.599776:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760709.599780:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545453fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.599790:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545453fd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.599810:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545453fd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.599817:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545453fd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760709.602690:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495734272:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41696us (42049us total) trans 4294972444 rc 0/0 00000100:00100000:2.0:1789760709.602705:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5156 00000100:00100000:2.0:1789760709.602716:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693799851286 time 693881687027 state 0x2 ref 0 00000100:00100000:3.0:1789760709.608141:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760709.608243:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495734400 00000100:00100000:0.0:1789760709.608252:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495734400 00000100:00100000:0.0:1789760709.608361:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693799851286 time 693881687027 state 0x0 ref 1 00000100:00100000:0.0:1789760709.608376:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693881947443 time 693881687027 state 0x1 ref 1 00000100:00100000:0.0:1789760709.608397:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693881947443 time 693929114194 state 0x0 ref 1 00000100:00100000:0.0:1789760709.608410:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495734400:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760709.608459:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21004288, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xa807 00010000:00010000:0.0:1789760709.608468:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545404 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.608488:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:0.0:1789760709.608493:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760709.608500:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545404 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.608514:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545404 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760709.654017:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972445 is committed 00002000:00010000:0.0:1789760709.654214:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21004288 -> 21008384 00010000:00010000:0.0:1789760709.654231:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545404 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.654244:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545404 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.654253:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545404 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.654271:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760709.654276:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545404 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.654327:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545404 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.654348:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545404 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.654356:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545404 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760709.654448:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495734400:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46035us (46312us total) trans 4294972445 rc 0/0 00000100:00100000:0.0:1789760709.654460:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5157 00000100:00100000:0.0:1789760709.654470:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693881947443 time 693929114194 state 0x2 ref 0 00000100:00100000:0.0:1789760709.658754:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760709.660722:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495734528 00000100:00100000:1.0:1789760709.660737:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495734528 00000100:00100000:1.0:1789760709.660801:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693881947443 time 693929114194 state 0x0 ref 1 00000100:00100000:1.0:1789760709.660818:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693929374610 time 693929114194 state 0x1 ref 1 00000100:00100000:1.0:1789760709.660850:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693929374610 time 693981567442 state 0x0 ref 1 00000100:00100000:1.0:1789760709.660864:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495734528:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760709.660913:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21008384, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xa807 00010000:00010000:1.0:1789760709.660922:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454540b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.660949:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:1.0:1789760709.660954:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760709.660961:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454540b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.660975:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454540b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760709.715202:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972446 is committed 00002000:00010000:1.0:1789760709.715326:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21008384 -> 21012480 00002000:00010000:1.0:1789760709.715331:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760708 -> 1789760709 00002000:00010000:1.0:1789760709.715333:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760708 -> 1789760709 00010000:00010000:1.0:1789760709.715347:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454540b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.715359:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454540b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.715367:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454540b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.715374:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760709.715379:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454540b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.715388:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454540b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.715408:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454540b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.715416:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454540b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760709.715508:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495734528:12345-192.168.203.9@tcp:4:dd.0 Request processed in 54641us (56758us total) trans 4294972446 rc 0/0 00000100:00100000:1.0:1789760709.715518:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5158 00000100:00100000:1.0:1789760709.715528:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693929374610 time 693981567442 state 0x2 ref 0 00000100:00100000:1.0:1789760709.719215:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760709.719318:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495734656 00000100:00100000:2.0:1789760709.719327:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495734656 00000100:00100000:2.0:1789760709.719385:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693929374610 time 693981567442 state 0x0 ref 1 00000100:00100000:2.0:1789760709.719402:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693981827858 time 693981567442 state 0x1 ref 1 00000100:00100000:2.0:1789760709.719426:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693981827858 time 694040143431 state 0x0 ref 1 00000100:00100000:2.0:1789760709.719438:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495734656:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760709.719492:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21012480, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:2.0:1789760709.719501:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545412 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.719518:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:2.0:1789760709.719523:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760709.719530:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545412 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.719544:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545412 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760709.785351:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21012480 -> 21016576 00010000:00010000:2.0:1789760709.785372:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545412 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.785386:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545412 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.785395:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545412 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.785402:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760709.785407:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545412 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.785417:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545412 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.785437:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545412 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.785444:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545412 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760709.785539:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495734656:12345-192.168.203.9@tcp:4:dd.0 Request processed in 66098us (66329us total) trans 4294972447 rc 0/0 00000100:00100000:2.0:1789760709.785550:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5159 00000100:00100000:2.0:1789760709.785559:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693981827858 time 694040143431 state 0x2 ref 0 00000001:00080000:1.0:1789760709.788388:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972447 is committed 00000100:00100000:3.0:1789760709.789796:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760709.789925:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495734784 00000100:00100000:2.0:1789760709.789947:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495734784 00000100:00100000:2.0:1789760709.789999:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 693981827858 time 694040143431 state 0x0 ref 1 00000100:00100000:2.0:1789760709.790014:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694040403847 time 694040143431 state 0x1 ref 1 00000100:00100000:2.0:1789760709.790042:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694040403847 time 694110759472 state 0x0 ref 1 00000100:00100000:2.0:1789760709.790054:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495734784:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760709.790104:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21016576, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:2.0:1789760709.790112:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545419 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.790129:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:2.0:1789760709.790134:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760709.790140:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545419 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.790153:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545419 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760709.823685:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21016576 -> 21020672 00010000:00010000:1.0:1789760709.823707:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545419 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.823722:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545419 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.823732:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545419 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.823740:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760709.823745:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545419 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.823754:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545419 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.823774:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545419 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.823782:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545419 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760709.823872:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495734784:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33814us (34080us total) trans 4294972448 rc 0/0 00000100:00100000:1.0:1789760709.823883:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5160 00000100:00100000:1.0:1789760709.823893:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694040403847 time 694110759472 state 0x2 ref 0 00000001:00080000:1.0:1789760709.823933:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972448 is committed 00000100:00100000:1.0:1789760709.830949:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760709.833453:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495734912 00000100:00100000:3.0:1789760709.833468:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495734912 00000100:00100000:3.0:1789760709.833529:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694040403847 time 694110759472 state 0x0 ref 1 00000100:00100000:3.0:1789760709.833546:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694111019888 time 694110759472 state 0x1 ref 1 00000100:00100000:3.0:1789760709.833577:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694111019888 time 694154293906 state 0x0 ref 1 00000100:00100000:3.0:1789760709.833590:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495734912:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760709.833640:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21020672, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:3.0:1789760709.833649:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545420 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.833666:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760709.833670:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760709.833676:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545420 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.833690:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545420 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760709.875038:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972449 is committed 00002000:00010000:3.0:1789760709.875149:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21020672 -> 21024768 00010000:00010000:3.0:1789760709.875165:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545420 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.875178:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545420 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.875188:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545420 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.875195:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760709.875200:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545420 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.875210:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545420 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.875229:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545420 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.875236:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545420 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760709.875385:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495734912:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41791us (44440us total) trans 4294972449 rc 0/0 00000100:00100000:3.0:1789760709.875396:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5161 00000100:00100000:3.0:1789760709.875407:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694111019888 time 694154293906 state 0x2 ref 0 00000100:00100000:2.0:1789760709.881379:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760709.881558:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495735040 00000100:00100000:0.0:1789760709.881567:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495735040 00000100:00100000:0.0:1789760709.881622:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694111019888 time 694154293906 state 0x0 ref 1 00000100:00100000:0.0:1789760709.881637:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694154554322 time 694154293906 state 0x1 ref 1 00000100:00100000:0.0:1789760709.881718:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694154554322 time 694202433756 state 0x0 ref 1 00000100:00100000:0.0:1789760709.881742:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495735040:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760709.881803:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21024768, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:0.0:1789760709.881812:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545427 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.881830:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:0.0:1789760709.881834:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760709.881842:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545427 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.881855:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545427 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760709.919108:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789760709.919132:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f04574700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520300800:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760709.919201:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760709.919307:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520300800 00000100:00100000:2.0:1789760709.919316:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520300800 00000100:00100000:2.0:1789760709.919376:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 187 00000100:00100000:2.0:1789760709.919385:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04577b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520300800:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760709.919410:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528023552 pending 1703936 free 3837788160 avail 3815768064 00000100:00100000:2.0:1789760709.919459:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04577b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520300800:12345-0@lo:13:osp-pre-0-0.0 Request processed in 74us (256us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760709.919467:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 187 00000100:00080000:2.0:1789760709.919489:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760709.919526:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f04574700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520300800:0@lo:13:osp-pre-0-0.0 00000001:00080000:2.0:1789760709.946230:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972450 is committed 00002000:00010000:0.0:1789760709.946499:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21024768 -> 21028864 00010000:00010000:0.0:1789760709.946514:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545427 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.946526:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545427 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.946535:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545427 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.946543:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94aa00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760709.946549:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545427 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.946557:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545427 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.946575:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545427 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.946583:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545427 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760709.946822:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495735040:12345-192.168.203.9@tcp:4:dd.0 Request processed in 65087us (65446us total) trans 4294972450 rc 0/0 00000100:00100000:0.0:1789760709.946833:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5162 00000100:00100000:0.0:1789760709.946843:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694154554322 time 694202433756 state 0x2 ref 0 00000100:00100000:1.0:1789760709.951881:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760709.951959:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495735168 00000100:00100000:2.0:1789760709.951970:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495735168 00000100:00100000:2.0:1789760709.952026:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694154554322 time 694202433756 state 0x0 ref 1 00000100:00100000:2.0:1789760709.952042:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694202694172 time 694202433756 state 0x1 ref 1 00000100:00100000:2.0:1789760709.952069:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694202694172 time 694272786436 state 0x0 ref 1 00000100:00100000:2.0:1789760709.952081:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495735168:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760709.952128:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21028864, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:2.0:1789760709.952137:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454542e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.952153:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:2.0:1789760709.952158:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760709.952164:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454542e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.952176:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454542e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760710.004326:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972451 is committed 00002000:00010000:2.0:1789760710.004796:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21028864 -> 21032960 00010000:00010000:2.0:1789760710.004814:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454542e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.004826:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454542e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.004835:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454542e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.004843:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760710.004848:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454542e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.004857:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454542e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.004879:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454542e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.004886:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454542e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760710.005000:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495735168:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52916us (53122us total) trans 4294972451 rc 0/0 00000100:00100000:2.0:1789760710.005010:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5163 00000100:00100000:2.0:1789760710.005020:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694202694172 time 694272786436 state 0x2 ref 0 00000100:00100000:1.0:1789760710.008950:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760710.009041:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495735296 00000100:00100000:3.0:1789760710.009050:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495735296 00000100:00100000:3.0:1789760710.009104:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694202694172 time 694272786436 state 0x0 ref 1 00000100:00100000:3.0:1789760710.009121:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694273046852 time 694272786436 state 0x1 ref 1 00000100:00100000:3.0:1789760710.009159:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694273046852 time 694329875322 state 0x0 ref 1 00000100:00100000:3.0:1789760710.009171:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495735296:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760710.009220:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21032960, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:3.0:1789760710.009230:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545435 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.009247:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760710.009252:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760710.009259:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545435 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.009317:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545435 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760710.077339:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972452 is committed 00002000:00010000:0.0:1789760710.077598:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21032960 -> 21037056 00010000:00010000:0.0:1789760710.077613:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545435 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.077625:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545435 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.077635:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545435 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.077643:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760710.077647:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545435 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.077657:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545435 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.077675:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545435 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.077687:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545435 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760710.077779:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495735296:12345-192.168.203.9@tcp:4:dd.0 Request processed in 68605us (68844us total) trans 4294972452 rc 0/0 00000100:00100000:0.0:1789760710.077790:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5164 00000100:00100000:0.0:1789760710.077800:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694273046852 time 694329875322 state 0x2 ref 0 00000100:00100000:3.0:1789760710.084538:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760710.084660:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495735424 00000100:00100000:0.0:1789760710.084669:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495735424 00000100:00100000:0.0:1789760710.084737:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694273046852 time 694329875322 state 0x0 ref 1 00000100:00100000:0.0:1789760710.084754:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694330135738 time 694329875322 state 0x1 ref 1 00000100:00100000:0.0:1789760710.084792:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694330135738 time 694405509211 state 0x0 ref 1 00000100:00100000:0.0:1789760710.084805:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495735424:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760710.084853:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21037056, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:0.0:1789760710.084862:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454543c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.084877:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:0.0:1789760710.084882:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760710.084889:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454543c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.084908:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454543c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760710.153934:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21037056 -> 21041152 00010000:00010000:0.0:1789760710.153957:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454543c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.153971:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454543c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.153980:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454543c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.153987:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94aa00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760710.153991:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454543c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.154000:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454543c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.154019:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454543c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.154028:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454543c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760710.154140:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495735424:12345-192.168.203.9@tcp:4:dd.0 Request processed in 69333us (69609us total) trans 4294972453 rc 0/0 00000100:00100000:0.0:1789760710.154151:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5165 00000100:00100000:0.0:1789760710.154161:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694330135738 time 694405509211 state 0x2 ref 0 00000001:00080000:3.0:1789760710.155220:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972453 is committed 00000100:00100000:1.0:1789760710.160053:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760710.160134:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495735552 00000100:00100000:2.0:1789760710.160144:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495735552 00000100:00100000:2.0:1789760710.160196:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694330135738 time 694405509211 state 0x0 ref 1 00000100:00100000:2.0:1789760710.160211:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694405769627 time 694405509211 state 0x1 ref 1 00000100:00100000:2.0:1789760710.160241:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694405769627 time 694480957300 state 0x0 ref 1 00000100:00100000:2.0:1789760710.160253:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495735552:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760710.160339:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21041152, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:2.0:1789760710.160349:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545443 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.160363:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:2.0:1789760710.160367:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760710.160373:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545443 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.160385:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545443 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760710.202495:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21041152 -> 21045248 00010000:00010000:3.0:1789760710.202519:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545443 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.202533:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545443 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.202542:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545443 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.202549:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760710.202554:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545443 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.202563:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545443 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.202589:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545443 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.202610:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545443 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760710.202712:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495735552:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42455us (42664us total) trans 4294972454 rc 0/0 00000100:00100000:3.0:1789760710.202724:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5166 00000100:00100000:3.0:1789760710.202734:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694405769627 time 694480957300 state 0x2 ref 0 00000001:00080000:0.0:1789760710.206512:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972454 is committed 00000100:00100000:2.0:1789760710.209067:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760710.209738:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495735680 00000100:00100000:0.0:1789760710.209749:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495735680 00000100:00100000:0.0:1789760710.209927:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694405769627 time 694480957300 state 0x0 ref 1 00000100:00100000:0.0:1789760710.209943:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694481217716 time 694480957300 state 0x1 ref 1 00000100:00100000:0.0:1789760710.209979:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694481217716 time 694530695341 state 0x0 ref 1 00000100:00100000:0.0:1789760710.209991:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495735680:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760710.210039:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21045248, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:0.0:1789760710.210049:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454544a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.210066:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:0.0:1789760710.210070:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760710.210077:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454544a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.210090:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454544a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760710.279926:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972455 is committed 00002000:00010000:2.0:1789760710.280004:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21045248 -> 21049344 00010000:00010000:2.0:1789760710.280020:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454544a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.280031:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454544a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.280040:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454544a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.280048:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94aa00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760710.280053:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454544a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.280062:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454544a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.280083:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454544a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.280090:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454544a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760710.280182:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495735680:12345-192.168.203.9@tcp:4:dd.0 Request processed in 70188us (71120us total) trans 4294972455 rc 0/0 00000100:00100000:2.0:1789760710.280191:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5167 00000100:00100000:2.0:1789760710.280200:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694481217716 time 694530695341 state 0x2 ref 0 00000100:00100000:2.0:1789760710.289786:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760710.289881:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495735808 00000100:00100000:0.0:1789760710.289891:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495735808 00000100:00100000:0.0:1789760710.289960:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694481217716 time 694530695341 state 0x0 ref 1 00000100:00100000:0.0:1789760710.289975:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694530955757 time 694530695341 state 0x1 ref 1 00000100:00100000:0.0:1789760710.290006:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694530955757 time 694610722741 state 0x0 ref 1 00000100:00100000:0.0:1789760710.290019:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495735808:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760710.290071:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21049344, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:0.0:1789760710.290080:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545451 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.290096:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:0.0:1789760710.290100:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760710.290107:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545451 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.290121:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545451 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760710.365165:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972456 is committed 00002000:00010000:2.0:1789760710.365472:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21049344 -> 21053440 00010000:00010000:2.0:1789760710.365486:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545451 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.365499:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545451 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.365508:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545451 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.365515:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760710.365519:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545451 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.365534:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545451 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.365553:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545451 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.365560:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545451 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760710.365732:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495735808:12345-192.168.203.9@tcp:4:dd.0 Request processed in 75710us (75949us total) trans 4294972456 rc 0/0 00000100:00100000:2.0:1789760710.365743:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5168 00000100:00100000:2.0:1789760710.365754:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694530955757 time 694610722741 state 0x2 ref 0 00000100:00100000:0.0:1789760710.370167:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760710.370267:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495735936 00000100:00100000:3.0:1789760710.370314:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495735936 00000100:00100000:3.0:1789760710.370381:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694530955757 time 694610722741 state 0x0 ref 1 00000100:00100000:3.0:1789760710.370397:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694610983157 time 694610722741 state 0x1 ref 1 00000100:00100000:3.0:1789760710.370428:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694610983157 time 694691144835 state 0x0 ref 1 00000100:00100000:3.0:1789760710.370440:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495735936:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760710.370485:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21053440, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:3.0:1789760710.370494:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545458 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.370508:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760710.370513:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760710.370519:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545458 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.370533:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545458 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760710.414103:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972457 is committed 00002000:00010000:2.0:1789760710.414126:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21053440 -> 21057536 00010000:00010000:2.0:1789760710.414141:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545458 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.414153:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545458 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.414162:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545458 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.414169:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760710.414175:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545458 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.414184:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545458 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.414204:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545458 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.414213:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545458 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760710.414370:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495735936:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43927us (44209us total) trans 4294972457 rc 0/0 00000100:00100000:2.0:1789760710.414381:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5169 00000100:00100000:2.0:1789760710.414391:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694610983157 time 694691144835 state 0x2 ref 0 00000100:00100000:1.0:1789760710.419768:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760710.419939:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495736064 00000100:00100000:2.0:1789760710.419950:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495736064 00000100:00100000:2.0:1789760710.420009:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694610983157 time 694691144835 state 0x0 ref 1 00000100:00100000:2.0:1789760710.420024:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694691405251 time 694691144835 state 0x1 ref 1 00000100:00100000:0.0:1789760710.420145:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694691405251 time 694740859756 state 0x0 ref 1 00000100:00100000:0.0:1789760710.420161:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495736064:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760710.420304:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21057536, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:0.0:1789760710.420314:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454545f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.420330:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:0.0:1789760710.420335:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760710.420341:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454545f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.420356:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454545f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760710.492473:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972458 is committed 00002000:00010000:3.0:1789760710.492614:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21057536 -> 21061632 00010000:00010000:3.0:1789760710.492629:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454545f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.492641:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454545f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.492650:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454545f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.492657:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760710.492662:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454545f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.492680:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454545f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.492701:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454545f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.492708:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454545f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760710.492912:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495736064:12345-192.168.203.9@tcp:4:dd.0 Request processed in 72749us (73154us total) trans 4294972458 rc 0/0 00000100:00100000:3.0:1789760710.492924:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5170 00000100:00100000:3.0:1789760710.492935:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694691405251 time 694740859756 state 0x2 ref 0 00010000:00010000:3.0:1789760710.495308:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760710.496714:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760710.496883:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495736192 00000100:00100000:2.0:1789760710.496893:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495736192 00000100:00100000:2.0:1789760710.496944:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694691405251 time 694740859756 state 0x0 ref 1 00000100:00100000:2.0:1789760710.496959:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694741120172 time 694740859756 state 0x1 ref 1 00000100:00100000:2.0:1789760710.496989:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694741120172 time 694817706320 state 0x0 ref 1 00000100:00100000:2.0:1789760710.497002:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495736192:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760710.497047:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21061632, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:2.0:1789760710.497056:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545466 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.497072:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:2.0:1789760710.497076:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760710.497083:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545466 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.497096:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545466 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760710.565268:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972459 is committed 00002000:00010000:2.0:1789760710.565646:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21061632 -> 21065728 00010000:00010000:2.0:1789760710.565661:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545466 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.565684:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545466 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.565694:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545466 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.565701:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760710.565706:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545466 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.565714:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545466 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.565741:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545466 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.565750:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545466 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760710.565883:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495736192:12345-192.168.203.9@tcp:4:dd.0 Request processed in 68878us (69172us total) trans 4294972459 rc 0/0 00000100:00100000:2.0:1789760710.565894:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5171 00000100:00100000:2.0:1789760710.565905:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694741120172 time 694817706320 state 0x2 ref 0 00000100:00100000:0.0:1789760710.571753:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760710.571889:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495736320 00000100:00100000:2.0:1789760710.571900:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495736320 00000100:00100000:2.0:1789760710.571957:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694741120172 time 694817706320 state 0x0 ref 1 00000100:00100000:2.0:1789760710.571973:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694817966736 time 694817706320 state 0x1 ref 1 00000100:00100000:2.0:1789760710.572003:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694817966736 time 694892720391 state 0x0 ref 1 00000100:00100000:2.0:1789760710.572016:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495736320:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760710.572062:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21065728, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:2.0:1789760710.572073:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454546d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.572089:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:2.0:1789760710.572094:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760710.572100:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454546d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.572114:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454546d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760710.651051:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972460 is committed 00002000:00010000:3.0:1789760710.651155:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21065728 -> 21069824 00010000:00010000:3.0:1789760710.651238:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454546d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.651357:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454546d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.651367:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454546d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.651373:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760710.651377:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454546d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.651386:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454546d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.651405:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454546d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.651416:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454546d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760710.651498:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495736320:12345-192.168.203.9@tcp:4:dd.0 Request processed in 79480us (79759us total) trans 4294972460 rc 0/0 00000100:00100000:3.0:1789760710.651508:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5172 00000100:00100000:3.0:1789760710.651519:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694817966736 time 694892720391 state 0x2 ref 0 00000100:00100000:1.0:1789760710.655144:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760710.655363:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495736448 00000100:00100000:3.0:1789760710.655373:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495736448 00000100:00100000:3.0:1789760710.655427:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694817966736 time 694892720391 state 0x0 ref 1 00000100:00100000:3.0:1789760710.655444:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694892980807 time 694892720391 state 0x1 ref 1 00000100:00100000:3.0:1789760710.655472:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694892980807 time 694976188848 state 0x0 ref 1 00000100:00100000:3.0:1789760710.655485:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495736448:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760710.655533:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21069824, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:3.0:1789760710.655543:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545474 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.655559:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760710.655564:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760710.655570:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545474 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.655590:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545474 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760710.707608:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21069824 -> 21073920 00002000:00010000:3.0:1789760710.707619:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760709 -> 1789760710 00002000:00010000:3.0:1789760710.707622:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760709 -> 1789760710 00010000:00010000:3.0:1789760710.707636:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545474 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.707649:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545474 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.707658:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545474 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.707666:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760710.707671:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545474 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.707681:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545474 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.707706:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545474 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.707719:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545474 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760710.707815:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495736448:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52327us (52677us total) trans 4294972461 rc 0/0 00000100:00100000:3.0:1789760710.707825:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5173 00000100:00100000:3.0:1789760710.707836:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694892980807 time 694976188848 state 0x2 ref 0 00000001:00080000:1.0:1789760710.709187:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972461 is committed 00000100:00100000:0.0:1789760710.711244:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760710.711702:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495736576 00000100:00100000:2.0:1789760710.711713:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495736576 00000100:00100000:2.0:1789760710.711770:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694892980807 time 694976188848 state 0x0 ref 1 00000100:00100000:2.0:1789760710.711785:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694976449264 time 694976188848 state 0x1 ref 1 00000100:00100000:2.0:1789760710.711814:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694976449264 time 695032530742 state 0x0 ref 1 00000100:00100000:2.0:1789760710.711826:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04292a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495736576:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760710.711873:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21073920, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:2.0:1789760710.711882:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454547b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.711899:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:2.0:1789760710.711904:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760710.711911:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454547b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.711926:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454547b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760710.756979:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972462 is committed 00002000:00010000:3.0:1789760710.757211:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21073920 -> 21078016 00010000:00010000:3.0:1789760710.757228:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454547b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.757241:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454547b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.757259:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454547b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.757267:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760710.757315:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454547b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.757326:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454547b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.757353:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454547b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.757361:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454547b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760710.757454:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04292a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495736576:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45625us (46214us total) trans 4294972462 rc 0/0 00000100:00100000:3.0:1789760710.757465:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5174 00000100:00100000:3.0:1789760710.757475:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694976449264 time 695032530742 state 0x2 ref 0 00000100:00100000:0.0:1789760710.761036:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760710.761122:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495736704 00000100:00100000:3.0:1789760710.761132:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495736704 00000100:00100000:3.0:1789760710.761192:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 694976449264 time 695032530742 state 0x0 ref 1 00000100:00100000:3.0:1789760710.761209:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695032791158 time 695032530742 state 0x1 ref 1 00000100:00100000:3.0:1789760710.761241:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695032791158 time 695081957684 state 0x0 ref 1 00000100:00100000:3.0:1789760710.761264:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04290700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495736704:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760710.761352:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21078016, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:3.0:1789760710.761361:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545482 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.761378:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760710.761383:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760710.761390:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545482 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.761405:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545482 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760710.827569:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972463 is committed 00002000:00010000:3.0:1789760710.827931:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21078016 -> 21082112 00010000:00010000:3.0:1789760710.827947:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545482 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.827960:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545482 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.827969:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545482 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.827975:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760710.827980:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545482 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.827989:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545482 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.828007:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545482 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.828014:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545482 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760710.828107:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04290700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495736704:12345-192.168.203.9@tcp:4:dd.0 Request processed in 66850us (67074us total) trans 4294972463 rc 0/0 00000100:00100000:3.0:1789760710.828117:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5175 00000100:00100000:3.0:1789760710.828127:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695032791158 time 695081957684 state 0x2 ref 0 00000100:00100000:1.0:1789760710.831652:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760710.835672:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495736832 00000100:00100000:3.0:1789760710.835688:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495736832 00000100:00100000:3.0:1789760710.835753:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695032791158 time 695081957684 state 0x0 ref 1 00000100:00100000:3.0:1789760710.835769:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695082218100 time 695081957684 state 0x1 ref 1 00000100:00100000:3.0:1789760710.835807:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695082218100 time 695156523853 state 0x0 ref 1 00000100:00100000:3.0:1789760710.835821:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495736832:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760710.835873:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21082112, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:3.0:1789760710.835883:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545489 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.835899:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760710.835904:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760710.835910:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545489 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.835924:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545489 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760710.875041:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972464 is committed 00002000:00010000:2.0:1789760710.875139:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21082112 -> 21086208 00010000:00010000:2.0:1789760710.875155:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545489 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.875168:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545489 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.875177:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545489 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.875184:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760710.875189:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545489 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.875208:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545489 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.875228:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545489 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.875236:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545489 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760710.875366:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495736832:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39543us (43780us total) trans 4294972464 rc 0/0 00000100:00100000:2.0:1789760710.875375:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5176 00000100:00100000:2.0:1789760710.875385:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695082218100 time 695156523853 state 0x2 ref 0 00000100:00100000:0.0:1789760710.878964:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760710.879051:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495736960 00000100:00100000:3.0:1789760710.879061:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495736960 00000100:00100000:3.0:1789760710.879115:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695082218100 time 695156523853 state 0x0 ref 1 00000100:00100000:3.0:1789760710.879131:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695156784269 time 695156523853 state 0x1 ref 1 00000100:00100000:3.0:1789760710.879165:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695156784269 time 695199881092 state 0x0 ref 1 00000100:00100000:3.0:1789760710.879177:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495736960:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760710.879222:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21086208, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:3.0:1789760710.879231:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545490 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.879257:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760710.879263:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760710.879307:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545490 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.879323:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545490 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760710.905789:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972465 is committed 00002000:00010000:1.0:1789760710.905963:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21086208 -> 21090304 00010000:00010000:1.0:1789760710.905978:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545490 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.905991:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545490 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.906001:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545490 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.906008:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760710.906012:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545490 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.906021:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545490 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.906041:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545490 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.906049:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545490 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760710.906142:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495736960:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26962us (27182us total) trans 4294972465 rc 0/0 00000100:00100000:1.0:1789760710.906152:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5177 00000100:00100000:1.0:1789760710.906163:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695156784269 time 695199881092 state 0x2 ref 0 00000100:00100000:0.0:1789760710.910759:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760710.910882:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495737088 00000100:00100000:3.0:1789760710.910891:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495737088 00000100:00100000:3.0:1789760710.910943:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695156784269 time 695199881092 state 0x0 ref 1 00000100:00100000:3.0:1789760710.910963:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695200141508 time 695199881092 state 0x1 ref 1 00000100:00100000:3.0:1789760710.911027:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695200141508 time 695231743119 state 0x0 ref 1 00000100:00100000:3.0:1789760710.911040:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04291180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495737088:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760710.911086:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21090304, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:3.0:1789760710.911095:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545497 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.911112:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760710.911116:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760710.911123:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545497 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.911138:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545497 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760710.957060:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972466 is committed 00002000:00010000:3.0:1789760710.960025:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21090304 -> 21094400 00010000:00010000:3.0:1789760710.960050:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545497 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.960064:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545497 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.960072:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545497 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.960083:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760710.960088:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545497 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.960097:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545497 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.960117:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545497 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.960125:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545497 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760710.960226:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04291180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495737088:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49182us (49471us total) trans 4294972466 rc 0/0 00000100:00100000:3.0:1789760710.960237:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5178 00000100:00100000:3.0:1789760710.960248:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695200141508 time 695231743119 state 0x2 ref 0 00000100:00100000:3.0:1789760710.966158:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760710.966474:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495737216 00000100:00100000:2.0:1789760710.966484:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495737216 00000100:00100000:2.0:1789760710.966537:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695200141508 time 695231743119 state 0x0 ref 1 00000100:00100000:2.0:1789760710.966552:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695232003535 time 695231743119 state 0x1 ref 1 00000100:00100000:2.0:1789760710.966592:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695232003535 time 695287309348 state 0x0 ref 1 00000100:00100000:2.0:1789760710.966605:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495737216:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760710.966652:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21094400, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:2.0:1789760710.966661:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454549e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.966678:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:2.0:1789760710.966682:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760710.966688:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454549e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.966700:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454549e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760711.035881:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972467 is committed 00002000:00010000:2.0:1789760711.036010:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21094400 -> 21098496 00010000:00010000:2.0:1789760711.036025:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454549e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.036040:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454549e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.036050:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454549e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.036057:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760711.036062:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454549e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.036073:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454549e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.036094:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454549e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.036102:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454549e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760711.036194:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495737216:12345-192.168.203.9@tcp:4:dd.0 Request processed in 69586us (70041us total) trans 4294972467 rc 0/0 00000100:00100000:2.0:1789760711.036206:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5179 00000100:00100000:2.0:1789760711.036216:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695232003535 time 695287309348 state 0x2 ref 0 00000100:00100000:0.0:1789760711.040780:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760711.041034:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495737344 00000100:00100000:1.0:1789760711.041046:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495737344 00000100:00100000:1.0:1789760711.041120:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695232003535 time 695287309348 state 0x0 ref 1 00000100:00100000:1.0:1789760711.041137:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695287569764 time 695287309348 state 0x1 ref 1 00000100:00100000:1.0:1789760711.041169:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695287569764 time 695361886372 state 0x0 ref 1 00000100:00100000:1.0:1789760711.041183:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495737344:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760711.041233:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21098496, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:1.0:1789760711.041243:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545454a5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.041312:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760711.041318:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760711.041325:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545454a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.041339:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545454a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760711.041861:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760711.128655:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972468 is committed 00002000:00010000:1.0:1789760711.128819:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21098496 -> 21102592 00010000:00010000:1.0:1789760711.128835:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545454a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.128854:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545454a5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.128862:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545454a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.128869:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760711.128874:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545454a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.128884:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545454a5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.128904:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545454a5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.128913:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545454a5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.129013:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495737344:12345-192.168.203.9@tcp:4:dd.0 Request processed in 87828us (88238us total) trans 4294972468 rc 0/0 00000100:00100000:1.0:1789760711.129023:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5180 00000100:00100000:1.0:1789760711.129034:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695287569764 time 695361886372 state 0x2 ref 0 00000100:00100000:1.0:1789760711.132691:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760711.132749:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495737472 00000100:00100000:3.0:1789760711.132758:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495737472 00000100:00100000:3.0:1789760711.132814:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695287569764 time 695361886372 state 0x0 ref 1 00000100:00100000:3.0:1789760711.132832:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695362146788 time 695361886372 state 0x1 ref 1 00000100:00100000:3.0:1789760711.132935:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695362146788 time 695453650047 state 0x0 ref 1 00000100:00100000:3.0:1789760711.132949:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495737472:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760711.132997:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21102592, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:3.0:1789760711.133007:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545454ac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.133022:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760711.133026:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760711.133032:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545454ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.133047:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545454ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760711.252494:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972469 is committed 00002000:00010000:1.0:1789760711.252863:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21102592 -> 21106688 00010000:00010000:1.0:1789760711.252882:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545454ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.252897:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545454ac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.252907:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545454ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.252915:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760711.252921:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545454ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.252932:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545454ac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.252957:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545454ac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.252964:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545454ac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.253066:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495737472:12345-192.168.203.9@tcp:4:dd.0 Request processed in 120115us (120379us total) trans 4294972469 rc 0/0 00000100:00100000:1.0:1789760711.253077:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5181 00000100:00100000:1.0:1789760711.253087:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695362146788 time 695453650047 state 0x2 ref 0 00000100:00100000:3.0:1789760711.257494:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760711.257624:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495737600 00000100:00100000:2.0:1789760711.257634:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495737600 00000100:00100000:2.0:1789760711.257689:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695362146788 time 695453650047 state 0x0 ref 1 00000100:00100000:2.0:1789760711.257706:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695453910463 time 695453650047 state 0x1 ref 1 00000100:00100000:2.0:1789760711.257740:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695453910463 time 695578455962 state 0x0 ref 1 00000100:00100000:2.0:1789760711.257752:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495737600:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760711.257803:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21106688, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:2.0:1789760711.257813:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545454b3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.257830:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:2.0:1789760711.257835:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760711.257841:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545454b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.257855:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545454b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760711.333156:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972470 is committed 00002000:00010000:1.0:1789760711.333505:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21106688 -> 21110784 00010000:00010000:1.0:1789760711.333523:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545454b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.333536:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545454b3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.333545:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545454b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.333553:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760711.333558:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545454b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.333568:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545454b3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.333820:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545454b3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.333830:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545454b3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.333971:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495737600:12345-192.168.203.9@tcp:4:dd.0 Request processed in 76216us (76482us total) trans 4294972470 rc 0/0 00000100:00100000:1.0:1789760711.333985:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5182 00000100:00100000:1.0:1789760711.333995:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695453910463 time 695578455962 state 0x2 ref 0 00000100:00100000:0.0:1789760711.338038:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760711.339737:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495737728 00000100:00100000:1.0:1789760711.339750:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495737728 00000100:00100000:1.0:1789760711.339811:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695453910463 time 695578455962 state 0x0 ref 1 00000100:00100000:1.0:1789760711.339827:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695578716378 time 695578455962 state 0x1 ref 1 00000100:00100000:1.0:1789760711.339865:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695578716378 time 695660581253 state 0x0 ref 1 00000100:00100000:1.0:1789760711.339878:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495737728:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760711.339930:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21110784, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:1.0:1789760711.339940:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545454ba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.339958:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760711.339963:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760711.339970:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545454ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.339985:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545454ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760711.391499:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972471 is committed 00002000:00010000:2.0:1789760711.391728:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21110784 -> 21114880 00010000:00010000:2.0:1789760711.391745:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545454ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.391758:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545454ba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.391767:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545454ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.391779:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760711.391784:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545454ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.391794:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545454ba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.391813:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545454ba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.391821:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545454ba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760711.391961:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495737728:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52080us (53927us total) trans 4294972471 rc 0/0 00000100:00100000:2.0:1789760711.391976:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5183 00000100:00100000:2.0:1789760711.391986:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695578716378 time 695660581253 state 0x2 ref 0 00000100:00100000:0.0:1789760711.395585:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760711.395927:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495737856 00000100:00100000:1.0:1789760711.395946:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495737856 00000100:00100000:1.0:1789760711.396004:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695578716378 time 695660581253 state 0x0 ref 1 00000100:00100000:1.0:1789760711.396020:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695660841669 time 695660581253 state 0x1 ref 1 00000100:00100000:1.0:1789760711.396094:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695660841669 time 695716809091 state 0x0 ref 1 00000100:00100000:1.0:1789760711.396108:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495737856:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760711.396155:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21114880, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:1.0:1789760711.396164:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545454c1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.396181:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760711.396186:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760711.396193:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545454c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.396207:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545454c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760711.435638:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972472 is committed 00002000:00010000:2.0:1789760711.435880:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21114880 -> 21118976 00010000:00010000:2.0:1789760711.435896:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545454c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.435908:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545454c1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.435918:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545454c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.435925:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760711.435930:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545454c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.435940:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545454c1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.435960:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545454c1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.435968:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545454c1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760711.436063:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495737856:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39952us (40486us total) trans 4294972472 rc 0/0 00000100:00100000:2.0:1789760711.436073:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5184 00000100:00100000:2.0:1789760711.436083:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695660841669 time 695716809091 state 0x2 ref 0 00000100:00100000:3.0:1789760711.444222:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760711.446567:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495737984 00000100:00100000:1.0:1789760711.446584:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495737984 00000100:00100000:1.0:1789760711.446656:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695660841669 time 695716809091 state 0x0 ref 1 00000100:00100000:1.0:1789760711.446672:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695717069507 time 695716809091 state 0x1 ref 1 00000100:00100000:1.0:1789760711.446707:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695717069507 time 695767423143 state 0x0 ref 1 00000100:00100000:1.0:1789760711.446721:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495737984:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760711.446767:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21118976, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:1.0:1789760711.446777:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454c8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.446794:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760711.446799:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760711.446806:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.446821:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760711.479627:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21118976 -> 21123072 00010000:00010000:1.0:1789760711.479646:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.479660:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454c8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.479670:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.479677:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760711.479682:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.479692:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.479712:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454c8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.479720:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454c8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.479810:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495737984:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33087us (35596us total) trans 4294972473 rc 0/0 00000100:00100000:1.0:1789760711.479820:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5185 00000100:00100000:1.0:1789760711.479829:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695717069507 time 695767423143 state 0x2 ref 0 00000001:00080000:3.0:1789760711.479962:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972473 is committed 00000100:00100000:1.0:1789760711.485666:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760711.485818:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495738112 00000100:00100000:3.0:1789760711.485829:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495738112 00000100:00100000:3.0:1789760711.485950:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695717069507 time 695767423143 state 0x0 ref 1 00000100:00100000:3.0:1789760711.485965:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695767683559 time 695767423143 state 0x1 ref 1 00000100:00100000:3.0:1789760711.486019:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695767683559 time 695806736086 state 0x0 ref 1 00000100:00100000:3.0:1789760711.486032:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495738112:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760711.486076:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21123072, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:3.0:1789760711.486086:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545454cf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.486103:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760711.486108:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760711.486115:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545454cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.486129:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545454cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.518568:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760711.549915:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21123072 -> 21127168 00010000:00010000:0.0:1789760711.549935:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545454cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.549949:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545454cf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.549957:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545454cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.549965:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760711.549970:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545454cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.552461:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545454cf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.552501:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545454cf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.552509:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545454cf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760711.552686:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495738112:12345-192.168.203.9@tcp:4:dd.0 Request processed in 66650us (68436us total) trans 4294972474 rc 0/0 00000100:00100000:0.0:1789760711.552697:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5186 00000100:00100000:0.0:1789760711.552707:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695767683559 time 695806736086 state 0x2 ref 0 00000001:00080000:1.0:1789760711.554609:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972474 is committed 00000100:00100000:2.0:1789760711.559526:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760711.560518:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495738240 00000100:00100000:1.0:1789760711.560529:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495738240 00000100:00100000:1.0:1789760711.560601:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695767683559 time 695806736086 state 0x0 ref 1 00000100:00100000:1.0:1789760711.560616:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695806996502 time 695806736086 state 0x1 ref 1 00000100:00100000:1.0:1789760711.560646:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695806996502 time 695881363217 state 0x0 ref 1 00000100:00100000:1.0:1789760711.560659:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495738240:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760711.560708:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21127168, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:1.0:1789760711.560717:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454d6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.560732:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760711.560736:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760711.560743:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.560755:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760711.610022:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972475 is committed 00002000:00010000:1.0:1789760711.610114:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21127168 -> 21131264 00010000:00010000:1.0:1789760711.610130:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.610144:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454d6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.610152:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.610160:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760711.610165:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.610173:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454d6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.610193:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454d6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.610201:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454d6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.610361:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495738240:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49700us (50840us total) trans 4294972475 rc 0/0 00000100:00100000:1.0:1789760711.610372:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5187 00000100:00100000:1.0:1789760711.610382:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695806996502 time 695881363217 state 0x2 ref 0 00000100:00100000:3.0:1789760711.615062:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760711.615574:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495738368 00000100:00100000:1.0:1789760711.615584:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495738368 00000100:00100000:1.0:1789760711.615640:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695806996502 time 695881363217 state 0x0 ref 1 00000100:00100000:1.0:1789760711.615655:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695881623633 time 695881363217 state 0x1 ref 1 00000100:00100000:1.0:1789760711.615788:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695881623633 time 695936503883 state 0x0 ref 1 00000100:00100000:1.0:1789760711.615802:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495738368:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760711.615853:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21131264, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:1.0:1789760711.615862:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454dd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.615877:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760711.615882:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760711.615889:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.615902:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760711.659681:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972476 is committed 00002000:00010000:1.0:1789760711.659834:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21131264 -> 21135360 00010000:00010000:1.0:1789760711.659850:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.659862:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454dd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.659871:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.659878:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760711.659882:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.659890:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454dd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.659909:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454dd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.659916:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454dd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.660048:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495738368:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44244us (45001us total) trans 4294972476 rc 0/0 00000100:00100000:1.0:1789760711.660059:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5188 00000100:00100000:1.0:1789760711.660068:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695881623633 time 695936503883 state 0x2 ref 0 00000100:00100000:0.0:1789760711.667935:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760711.668047:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495738496 00000100:00100000:3.0:1789760711.668056:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495738496 00000100:00100000:3.0:1789760711.668121:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695881623633 time 695936503883 state 0x0 ref 1 00000100:00100000:3.0:1789760711.668136:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695936764299 time 695936503883 state 0x1 ref 1 00000100:00100000:3.0:1789760711.668169:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695936764299 time 695988885943 state 0x0 ref 1 00000100:00100000:3.0:1789760711.668180:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495738496:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760711.668230:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21135360, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:3.0:1789760711.668239:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545454e4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.668578:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760711.668584:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760711.668592:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545454e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.668608:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545454e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760711.712331:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972477 is committed 00002000:00010000:2.0:1789760711.712919:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21135360 -> 21139456 00002000:00010000:2.0:1789760711.712925:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760710 -> 1789760711 00002000:00010000:2.0:1789760711.712928:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760710 -> 1789760711 00010000:00010000:2.0:1789760711.712941:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545454e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.712953:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545454e4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.712962:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545454e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.712969:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760711.712974:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545454e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.712983:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545454e4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.713011:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545454e4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.713020:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545454e4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760711.713477:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495738496:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45292us (45545us total) trans 4294972477 rc 0/0 00000100:00100000:2.0:1789760711.713489:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5189 00000100:00100000:2.0:1789760711.713499:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695936764299 time 695988885943 state 0x2 ref 0 00000100:00100000:2.0:1789760711.717174:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760711.717696:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495738624 00000100:00100000:1.0:1789760711.717707:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495738624 00000100:00100000:3.0:1789760711.718042:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695936764299 time 695988885943 state 0x0 ref 1 00000100:00100000:3.0:1789760711.718059:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695989146359 time 695988885943 state 0x1 ref 1 00000100:00100000:3.0:1789760711.718091:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695989146359 time 696038808020 state 0x0 ref 1 00000100:00100000:3.0:1789760711.718103:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495738624:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760711.718157:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21139456, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:3.0:1789760711.718168:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545454eb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.718184:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:3.0:1789760711.718189:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760711.718195:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545454eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.718210:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545454eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760711.805245:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21139456 -> 21143552 00010000:00010000:1.0:1789760711.805274:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545454eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.805318:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545454eb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.805327:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545454eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.805336:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760711.805340:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545454eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.805348:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545454eb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.805366:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545454eb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.805375:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545454eb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.805477:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495738624:12345-192.168.203.9@tcp:4:dd.0 Request processed in 87357us (88292us total) trans 4294972478 rc 0/0 00000100:00100000:1.0:1789760711.805496:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5190 00000100:00100000:1.0:1789760711.805508:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695989146359 time 696038808020 state 0x2 ref 0 00000001:00080000:1.0:1789760711.805551:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972478 is committed 00000100:00100000:3.0:1789760711.816467:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760711.816555:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495738752 00000100:00100000:1.0:1789760711.816565:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495738752 00000100:00100000:1.0:1789760711.816625:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 695989146359 time 696038808020 state 0x0 ref 1 00000100:00100000:1.0:1789760711.816639:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696039068436 time 696038808020 state 0x1 ref 1 00000100:00100000:1.0:1789760711.816674:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696039068436 time 696137391032 state 0x0 ref 1 00000100:00100000:1.0:1789760711.816697:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495738752:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760711.816748:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21143552, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:1.0:1789760711.816757:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454f2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.816783:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760711.816788:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760711.816795:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.816810:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760711.882842:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972479 is committed 00002000:00010000:1.0:1789760711.882952:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21143552 -> 21147648 00010000:00010000:1.0:1789760711.882967:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.882992:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454f2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.883002:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.883010:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760711.883015:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.883024:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454f2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.883044:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454f2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.883052:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545454f2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.883154:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495738752:12345-192.168.203.9@tcp:4:dd.0 Request processed in 66464us (66690us total) trans 4294972479 rc 0/0 00000100:00100000:1.0:1789760711.883165:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5191 00000100:00100000:1.0:1789760711.883176:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696039068436 time 696137391032 state 0x2 ref 0 00000100:00100000:3.0:1789760711.890382:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760711.890688:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495738880 00000100:00100000:2.0:1789760711.890699:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495738880 00000100:00100000:2.0:1789760711.890754:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696039068436 time 696137391032 state 0x0 ref 1 00000100:00100000:2.0:1789760711.890770:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696137651448 time 696137391032 state 0x1 ref 1 00000100:00100000:2.0:1789760711.890864:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696137651448 time 696211579285 state 0x0 ref 1 00000100:00100000:2.0:1789760711.890877:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495738880:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760711.890926:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21147648, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:2.0:1789760711.890935:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545454f9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.890957:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:2.0:1789760711.890961:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760711.890968:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545454f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.891235:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545454f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760711.957506:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972480 is committed 00002000:00010000:1.0:1789760711.957610:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21147648 -> 21151744 00010000:00010000:1.0:1789760711.957625:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545454f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.957638:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545454f9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.957647:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545454f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.957654:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760711.957659:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545454f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.957668:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545454f9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.957686:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545454f9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.957698:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545454f9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.957788:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495738880:12345-192.168.203.9@tcp:4:dd.0 Request processed in 66908us (67412us total) trans 4294972480 rc 0/0 00000100:00100000:1.0:1789760711.957798:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5192 00000100:00100000:1.0:1789760711.957808:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696137651448 time 696211579285 state 0x2 ref 0 00000100:00100000:0.0:1789760711.961603:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760711.961687:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495739008 00000100:00100000:1.0:1789760711.961697:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495739008 00000100:00100000:1.0:1789760711.961754:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696137651448 time 696211579285 state 0x0 ref 1 00000100:00100000:1.0:1789760711.961770:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696211839701 time 696211579285 state 0x1 ref 1 00000100:00100000:1.0:1789760711.961800:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696211839701 time 696282517077 state 0x0 ref 1 00000100:00100000:1.0:1789760711.961812:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495739008:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760711.961860:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21151744, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:1.0:1789760711.961869:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545500 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.961886:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760711.961891:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760711.961896:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545500 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.961916:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545500 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760711.972513:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:59c4e264-9b75-482b-8813-3d1dc124f384:3317:1876699520300928:0@lo:400:kworker.0 00000100:00100000:2.0:1789760711.972583:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760711.972632:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3317:1876699520301440:0@lo:400:kworker.0 00000100:00100000:2.0:1789760711.972658:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760711.972683:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1789760711.972689:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3317:1876699520301056:0@lo:400:kworker.0 00000100:00100000:2.0:1789760711.972711:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760711.972729:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3317:1876699520301568:0@lo:400:kworker.0 00000100:00100000:2.0:1789760711.972752:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760711.972780:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520301184:0@lo:400:kworker.0 00000100:00100000:2.0:1789760711.972803:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760711.972820:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789760711.972825:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520301312:0@lo:400:kworker.0 00000100:00100000:2.0:1789760711.972846:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760711.972875:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520300928 00000100:00100000:2.0:1789760711.972882:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520300928 00000100:00100000:2.0:1789760711.972925:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 389 00000100:00100000:2.0:1789760711.972933:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929012314450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:59c4e264-9b75-482b-8813-3d1dc124f384+19:3317:x1876699520300928:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760711.972979:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929012314450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:59c4e264-9b75-482b-8813-3d1dc124f384+19:3317:x1876699520300928:12345-0@lo:400:kworker.0 Request processed in 46us (396us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760711.972986:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 389 00000100:00100000:2.0:1789760711.973017:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520301440 00000100:00100000:2.0:1789760711.973021:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520301440 00000100:00100000:2.0:1789760711.973052:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520301056 00000100:00100000:2.0:1789760711.973055:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520301056 00000100:00100000:2.0:1789760711.973072:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520301184 00000100:00100000:2.0:1789760711.973074:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520301184 00000100:00100000:2.0:1789760711.973089:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 452 00000100:00100000:2.0:1789760711.973095:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1876699520301440:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760711.973129:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1876699520301440:12345-0@lo:400:kworker.0 Request processed in 33us (470us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760711.973135:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 452 00000100:00100000:2.0:1789760711.973147:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 453 00000100:00100000:2.0:1789760711.973152:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1876699520301056:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760711.973177:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1876699520301056:12345-0@lo:400:kworker.0 Request processed in 24us (465us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760711.973183:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 453 00000100:00100000:2.0:1789760711.973192:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 454 00000100:00100000:2.0:1789760711.973198:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520301184:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760711.973245:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520301184:12345-0@lo:400:kworker.0 Request processed in 29us (423us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760711.973251:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 454 00000100:00100000:2.0:1789760711.973310:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520301568 00000100:00100000:2.0:1789760711.973314:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520301568 00000100:00100000:2.0:1789760711.973336:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520301312 00000100:00100000:2.0:1789760711.973339:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520301312 00000100:00100000:2.0:1789760711.973356:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 212 00000100:00100000:2.0:1789760711.973363:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3317:x1876699520301568:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760711.973393:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3317:x1876699520301568:12345-0@lo:400:kworker.0 Request processed in 31us (640us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760711.973399:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 212 00000100:00100000:2.0:1789760711.973411:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 213 00000100:00100000:2.0:1789760711.973417:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520301312:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760711.973441:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520301312:12345-0@lo:400:kworker.0 Request processed in 25us (594us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760711.973447:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 213 00000100:00080000:2.0:1789760711.973471:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789760711.973475:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760711.973477:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760711.973522:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520301312:0@lo:400:kworker.0 00000100:00100000:2.0:1789760711.973533:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520301184:0@lo:400:kworker.0 00000100:00100000:2.0:1789760711.973633:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3317:1876699520301056:0@lo:400:kworker.0 00000100:00100000:2.0:1789760711.973642:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3317:1876699520301568:0@lo:400:kworker.0 00000100:00100000:2.0:1789760711.973651:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:59c4e264-9b75-482b-8813-3d1dc124f384:3317:1876699520300928:0@lo:400:kworker.0 00000100:00100000:2.0:1789760711.973660:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3317:1876699520301440:0@lo:400:kworker.0 00000001:00080000:2.0:1789760712.043962:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972481 is committed 00002000:00010000:1.0:1789760712.044499:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21151744 -> 21155840 00010000:00010000:1.0:1789760712.044519:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545500 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.044531:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545500 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.044541:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545500 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.044549:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760712.044554:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545500 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.044564:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545500 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.044584:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545500 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.044593:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545500 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760712.044743:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495739008:12345-192.168.203.9@tcp:4:dd.0 Request processed in 82927us (83146us total) trans 4294972481 rc 0/0 00000100:00100000:1.0:1789760712.044754:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5193 00000100:00100000:1.0:1789760712.044764:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696211839701 time 696282517077 state 0x2 ref 0 00000100:00100000:0.0:1789760712.050041:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760712.051318:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495739136 00000100:00100000:1.0:1789760712.051332:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495739136 00000100:00100000:1.0:1789760712.051410:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696211839701 time 696282517077 state 0x0 ref 1 00000100:00100000:1.0:1789760712.051438:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696282777493 time 696282517077 state 0x1 ref 1 00000100:00100000:1.0:1789760712.051479:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696282777493 time 696372195862 state 0x0 ref 1 00000100:00100000:1.0:1789760712.051493:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495739136:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760712.051544:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21155840, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:1.0:1789760712.051553:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545507 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.051570:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760712.051575:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760712.052154:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545507 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.052175:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545507 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760712.090293:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972482 is committed 00002000:00010000:2.0:1789760712.091389:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21155840 -> 21159936 00010000:00010000:2.0:1789760712.091407:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545507 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.091421:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545507 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.091441:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545507 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.091448:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760712.091457:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545507 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.091467:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545507 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.091496:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545507 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.091504:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545507 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.091807:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495739136:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40312us (41772us total) trans 4294972482 rc 0/0 00000100:00100000:3.0:1789760712.091818:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5194 00000100:00100000:3.0:1789760712.091889:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696282777493 time 696372195862 state 0x2 ref 0 00000100:00100000:1.0:1789760712.096471:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760712.096718:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495739264 00000100:00100000:3.0:1789760712.096728:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495739264 00000100:00100000:3.0:1789760712.096782:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696282777493 time 696372195862 state 0x0 ref 1 00000100:00100000:3.0:1789760712.096797:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696372456278 time 696372195862 state 0x1 ref 1 00000100:00100000:3.0:1789760712.096824:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696372456278 time 696417541376 state 0x0 ref 1 00000100:00100000:3.0:1789760712.096836:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495739264:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760712.096880:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21159936, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:3.0:1789760712.096889:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454550e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.096906:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:3.0:1789760712.096910:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760712.096916:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454550e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.096935:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454550e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760712.121194:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972483 is committed 00002000:00010000:1.0:1789760712.121375:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21159936 -> 21164032 00010000:00010000:1.0:1789760712.121390:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454550e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.121404:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454550e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.121413:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454550e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.121421:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760712.121426:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454550e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.121435:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454550e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.121454:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454550e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.121462:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454550e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760712.121589:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495739264:12345-192.168.203.9@tcp:4:dd.0 Request processed in 24750us (25122us total) trans 4294972483 rc 0/0 00000100:00100000:1.0:1789760712.121599:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5195 00000100:00100000:1.0:1789760712.121609:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696372456278 time 696417541376 state 0x2 ref 0 00000100:00100000:2.0:1789760712.125223:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760712.125353:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495739392 00000100:00100000:1.0:1789760712.125362:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495739392 00000100:00100000:1.0:1789760712.125412:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696372456278 time 696417541376 state 0x0 ref 1 00000100:00100000:1.0:1789760712.125426:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696417801792 time 696417541376 state 0x1 ref 1 00000100:00100000:1.0:1789760712.125456:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696417801792 time 696446172868 state 0x0 ref 1 00000100:00100000:1.0:1789760712.125468:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495739392:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760712.125516:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21164032, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:1.0:1789760712.125525:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545515 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.125541:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760712.125546:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760712.125553:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545515 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.125566:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545515 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760712.168277:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972484 is committed 00002000:00010000:1.0:1789760712.168494:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21164032 -> 21168128 00010000:00010000:1.0:1789760712.168510:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545515 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.168523:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545515 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.168534:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545515 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.168542:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760712.168547:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545515 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.168557:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545515 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.168641:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545515 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.168651:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545515 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760712.168801:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495739392:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43329us (43582us total) trans 4294972484 rc 0/0 00000100:00100000:1.0:1789760712.168816:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5196 00000100:00100000:1.0:1789760712.168826:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696417801792 time 696446172868 state 0x2 ref 0 00000100:00100000:0.0:1789760712.172446:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760712.172611:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495739520 00000100:00100000:3.0:1789760712.172620:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495739520 00000100:00100000:3.0:1789760712.172671:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696417801792 time 696446172868 state 0x0 ref 1 00000100:00100000:3.0:1789760712.172686:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696446433284 time 696446172868 state 0x1 ref 1 00000100:00100000:3.0:1789760712.172725:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696446433284 time 696493441840 state 0x0 ref 1 00000100:00100000:3.0:1789760712.172736:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495739520:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760712.172787:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21168128, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:3.0:1789760712.172796:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454551c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.172813:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:3.0:1789760712.172817:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760712.172824:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454551c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.172838:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454551c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760712.203807:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21168128 -> 21172224 00010000:00010000:1.0:1789760712.203828:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454551c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.203842:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454551c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.203851:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454551c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.203859:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760712.203868:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454551c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.203878:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454551c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760712.203897:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972485 is committed 00010000:00010000:1.0:1789760712.203898:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454551c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.203905:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454551c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760712.204094:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495739520:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31354us (31652us total) trans 4294972485 rc 0/0 00000100:00100000:1.0:1789760712.204106:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5197 00000100:00100000:1.0:1789760712.204116:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696446433284 time 696493441840 state 0x2 ref 0 00000100:00100000:2.0:1789760712.208256:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760712.208675:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495739648 00000100:00100000:1.0:1789760712.208685:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495739648 00000100:00100000:1.0:1789760712.208745:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696446433284 time 696493441840 state 0x0 ref 1 00000100:00100000:1.0:1789760712.208761:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696493702256 time 696493441840 state 0x1 ref 1 00000100:00100000:1.0:1789760712.208790:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696493702256 time 696529506400 state 0x0 ref 1 00000100:00100000:1.0:1789760712.208803:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495739648:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760712.208852:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21172224, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:1.0:1789760712.208861:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545523 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.208877:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760712.208881:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760712.208888:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545523 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.208903:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545523 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760712.244657:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972486 is committed 00002000:00010000:0.0:1789760712.244722:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21172224 -> 21176320 00010000:00010000:0.0:1789760712.244737:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545523 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.244749:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545523 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.244758:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545523 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.244766:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760712.244771:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545523 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.244779:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545523 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.244804:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545523 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.244813:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545523 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760712.244972:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495739648:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36164us (36722us total) trans 4294972486 rc 0/0 00000100:00100000:0.0:1789760712.244985:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5198 00000100:00100000:0.0:1789760712.244996:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696493702256 time 696529506400 state 0x2 ref 0 00000100:00100000:2.0:1789760712.252004:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760712.252514:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495739776 00000100:00100000:1.0:1789760712.252525:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495739776 00000100:00100000:1.0:1789760712.252600:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696493702256 time 696529506400 state 0x0 ref 1 00000100:00100000:1.0:1789760712.252615:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696529766816 time 696529506400 state 0x1 ref 1 00000100:00100000:1.0:1789760712.252654:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696529766816 time 696573371248 state 0x0 ref 1 00000100:00100000:1.0:1789760712.252667:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495739776:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760712.252718:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21176320, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:1.0:1789760712.252728:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454552a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.252743:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760712.252747:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760712.252752:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454552a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.252765:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454552a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760712.289676:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972487 is committed 00002000:00010000:3.0:1789760712.290000:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21176320 -> 21180416 00010000:00010000:3.0:1789760712.290015:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454552a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.290027:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454552a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.290036:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454552a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.290044:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760712.290048:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454552a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.290057:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454552a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.290075:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454552a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.290082:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454552a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.290178:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495739776:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37509us (38178us total) trans 4294972487 rc 0/0 00000100:00100000:3.0:1789760712.290192:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5199 00000100:00100000:3.0:1789760712.290202:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696529766816 time 696573371248 state 0x2 ref 0 00000100:00100000:3.0:1789760712.294146:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760712.294221:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495739904 00000100:00100000:1.0:1789760712.294230:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495739904 00000100:00100000:1.0:1789760712.294610:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696529766816 time 696573371248 state 0x0 ref 1 00000100:00100000:1.0:1789760712.294627:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696573631664 time 696573371248 state 0x1 ref 1 00000100:00100000:1.0:1789760712.295087:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696573631664 time 696615799973 state 0x0 ref 1 00000100:00100000:1.0:1789760712.295105:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495739904:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760712.295220:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21180416, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:1.0:1789760712.295230:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545531 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.295246:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760712.295251:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760712.295257:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545531 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.295300:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545531 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760712.321743:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972488 is committed 00002000:00010000:3.0:1789760712.322003:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21180416 -> 21184512 00010000:00010000:3.0:1789760712.322018:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545531 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.322031:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545531 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.322040:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545531 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.322048:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760712.322053:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545531 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.322062:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545531 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.322082:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545531 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.322091:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545531 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.322175:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495739904:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27070us (28033us total) trans 4294972488 rc 0/0 00000100:00100000:3.0:1789760712.322188:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5200 00000100:00100000:3.0:1789760712.322198:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696573631664 time 696615799973 state 0x2 ref 0 00000100:00100000:2.0:1789760712.326227:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760712.326397:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495740032 00000100:00100000:0.0:1789760712.326406:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495740032 00000100:00100000:0.0:1789760712.326453:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696573631664 time 696615799973 state 0x0 ref 1 00000100:00100000:0.0:1789760712.326468:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696616060389 time 696615799973 state 0x1 ref 1 00000100:00100000:0.0:1789760712.326527:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696616060389 time 696647244242 state 0x0 ref 1 00000100:00100000:0.0:1789760712.326540:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495740032:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760712.326585:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21184512, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:0.0:1789760712.326593:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545538 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.326614:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760712.326619:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760712.326625:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545538 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.326638:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545538 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760712.383534:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21184512 -> 21188608 00010000:00010000:0.0:1789760712.383556:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545538 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.383580:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545538 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.383589:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545538 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.383596:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760712.383600:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545538 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.383609:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545538 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.383628:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545538 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.383636:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545538 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760712.383725:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495740032:12345-192.168.203.9@tcp:4:dd.0 Request processed in 57182us (57503us total) trans 4294972489 rc 0/0 00000100:00100000:0.0:1789760712.383735:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5201 00000100:00100000:0.0:1789760712.383745:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696616060389 time 696647244242 state 0x2 ref 0 00000001:00080000:0.0:1789760712.384706:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972489 is committed 00000100:00100000:2.0:1789760712.395715:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760712.395802:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495740160 00000100:00100000:3.0:1789760712.395812:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495740160 00000100:00100000:3.0:1789760712.395869:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696616060389 time 696647244242 state 0x0 ref 1 00000100:00100000:3.0:1789760712.395885:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696647504658 time 696647244242 state 0x1 ref 1 00000100:00100000:3.0:1789760712.395921:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696647504658 time 696716637208 state 0x0 ref 1 00000100:00100000:3.0:1789760712.395934:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495740160:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760712.395985:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21188608, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:3.0:1789760712.395994:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454553f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.396012:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:3.0:1789760712.396016:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760712.396023:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454553f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.396038:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454553f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760712.482620:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972490 is committed 00002000:00010000:1.0:1789760712.482790:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21188608 -> 21192704 00010000:00010000:1.0:1789760712.482813:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454553f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.482826:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454553f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.482834:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454553f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.482845:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760712.482850:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454553f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.482859:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454553f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.482879:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454553f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.482886:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454553f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760712.482999:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495740160:12345-192.168.203.9@tcp:4:dd.0 Request processed in 87061us (87288us total) trans 4294972490 rc 0/0 00000100:00100000:1.0:1789760712.483010:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5202 00000100:00100000:1.0:1789760712.483020:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696647504658 time 696716637208 state 0x2 ref 0 00000100:00100000:3.0:1789760712.484906:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789760712.484926:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f0276b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520301696:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760712.484986:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760712.485053:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520301696 00000100:00100000:3.0:1789760712.485061:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520301696 00000100:00100000:3.0:1789760712.485112:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 188 00000100:00100000:3.0:1789760712.485122:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520301696:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760712.485155:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760712.485207:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520301696:12345-0@lo:13:osp-pre-1-0.0 Request processed in 87us (219us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760712.485214:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 188 00000100:00100000:1.0:1789760712.485365:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f0276b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520301696:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760712.494323:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760712.494426:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495740288 00000100:00100000:3.0:1789760712.494436:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495740288 00000100:00100000:3.0:1789760712.494504:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696647504658 time 696716637208 state 0x0 ref 1 00000100:00100000:3.0:1789760712.494519:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696716897624 time 696716637208 state 0x1 ref 1 00000100:00100000:3.0:1789760712.494551:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696716897624 time 696815267529 state 0x0 ref 1 00000100:00100000:3.0:1789760712.494563:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495740288:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760712.494611:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21192704, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:3.0:1789760712.494620:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545546 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.494635:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:3.0:1789760712.494640:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760712.494645:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545546 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.494676:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545546 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.543954:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760712.558157:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21192704 -> 21196800 00010000:00010000:1.0:1789760712.558177:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545546 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.558191:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545546 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.558205:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545546 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.558212:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760712.558217:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545546 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.558226:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545546 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.558265:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545546 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.558274:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545546 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760712.558393:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495740288:12345-192.168.203.9@tcp:4:dd.0 Request processed in 63828us (64075us total) trans 4294972491 rc 0/0 00000100:00100000:1.0:1789760712.558403:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5203 00000100:00100000:1.0:1789760712.558412:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696716897624 time 696815267529 state 0x2 ref 0 00000001:00080000:0.0:1789760712.559985:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972491 is committed 00000100:00100000:0.0:1789760712.577767:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760712.577919:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495740416 00000100:00100000:2.0:1789760712.577929:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495740416 00000100:00100000:2.0:1789760712.577981:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696716897624 time 696815267529 state 0x0 ref 1 00000100:00100000:2.0:1789760712.577996:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696815527945 time 696815267529 state 0x1 ref 1 00000100:00100000:2.0:1789760712.578028:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696815527945 time 696898744817 state 0x0 ref 1 00000100:00100000:2.0:1789760712.578039:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f873480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495740416:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760712.578084:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21196800, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:2.0:1789760712.578093:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454554d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.578108:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:2.0:1789760712.578112:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760712.578117:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454554d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.578130:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454554d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760712.618068:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972492 is committed 00002000:00010000:1.0:1789760712.618187:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21196800 -> 21200896 00010000:00010000:1.0:1789760712.618201:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454554d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.618221:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454554d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.618230:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454554d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.618237:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760712.618242:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454554d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.618251:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454554d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.618307:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454554d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.618316:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454554d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760712.618405:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f873480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495740416:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40363us (40643us total) trans 4294972492 rc 0/0 00000100:00100000:1.0:1789760712.618415:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5204 00000100:00100000:1.0:1789760712.618426:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696815527945 time 696898744817 state 0x2 ref 0 00000100:00100000:3.0:1789760712.620994:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760712.621034:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495740544 00000100:00100000:1.0:1789760712.621043:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495740544 00000100:00100000:1.0:1789760712.621094:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696815527945 time 696898744817 state 0x0 ref 1 00000100:00100000:1.0:1789760712.621110:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696899005233 time 696898744817 state 0x1 ref 1 00000100:00100000:1.0:1789760712.621140:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696899005233 time 696941857232 state 0x0 ref 1 00000100:00100000:1.0:1789760712.621151:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495740544:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760712.621204:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21200896, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:1.0:1789760712.621212:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545554 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.621229:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760712.621234:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760712.621239:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545554 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.621253:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545554 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760712.663952:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972493 is committed 00002000:00010000:1.0:1789760712.664126:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21200896 -> 21204992 00002000:00010000:1.0:1789760712.664130:0:6714:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760711 -> 1789760712 00002000:00010000:1.0:1789760712.664133:0:6714:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760711 -> 1789760712 00010000:00010000:1.0:1789760712.664146:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545554 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.664158:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545554 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.664169:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545554 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.664177:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760712.664183:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545554 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.664192:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545554 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.664211:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545554 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.664218:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545554 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760712.665119:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495740544:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43962us (44125us total) trans 4294972493 rc 0/0 00000100:00100000:0.0:1789760712.665128:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5205 00000100:00100000:0.0:1789760712.665136:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696899005233 time 696941857232 state 0x2 ref 0 00000100:00100000:2.0:1789760712.671706:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760712.671796:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495740672 00000100:00100000:1.0:1789760712.671807:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495740672 00000100:00100000:1.0:1789760712.671860:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696899005233 time 696941857232 state 0x0 ref 1 00000100:00100000:1.0:1789760712.671876:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696942117648 time 696941857232 state 0x1 ref 1 00000100:00100000:1.0:1789760712.671909:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696942117648 time 696992625671 state 0x0 ref 1 00000100:00100000:1.0:1789760712.671921:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495740672:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760712.671968:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21204992, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:1.0:1789760712.671975:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454555b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.671988:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760712.671991:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760712.671995:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454555b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.672010:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454555b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760712.776442:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21204992 -> 21209088 00010000:00010000:3.0:1789760712.776835:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454555b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.776850:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454555b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.776860:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454555b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.776868:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760712.776874:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454555b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.776884:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454555b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.776909:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454555b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.776918:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454555b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.777028:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495740672:12345-192.168.203.9@tcp:4:dd.0 Request processed in 105103us (105325us total) trans 4294972494 rc 0/0 00000100:00100000:3.0:1789760712.777041:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5206 00000100:00100000:3.0:1789760712.777054:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696942117648 time 696992625671 state 0x2 ref 0 00000001:00080000:3.0:1789760712.777111:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972494 is committed 00000100:00100000:2.0:1789760712.784170:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760712.784528:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495740800 00000100:00100000:1.0:1789760712.784540:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495740800 00000100:00100000:1.0:1789760712.784607:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696942117648 time 696992625671 state 0x0 ref 1 00000100:00100000:1.0:1789760712.784624:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696992886087 time 696992625671 state 0x1 ref 1 00000100:00100000:1.0:1789760712.784671:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696992886087 time 697105386334 state 0x0 ref 1 00000100:00100000:1.0:1789760712.784697:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495740800:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760712.784763:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21209088, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:1.0:1789760712.784775:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545562 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.784794:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:1.0:1789760712.784801:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760712.784806:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545562 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.785566:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545562 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760712.866630:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972495 is committed 00002000:00010000:0.0:1789760712.866970:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21209088 -> 21213184 00010000:00010000:0.0:1789760712.866987:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545562 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.867001:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545562 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.867010:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545562 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.867018:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760712.867023:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545562 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.867033:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545562 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.867054:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545562 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.867062:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545562 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760712.867164:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495740800:12345-192.168.203.9@tcp:4:dd.0 Request processed in 82477us (83001us total) trans 4294972495 rc 0/0 00000100:00100000:0.0:1789760712.867176:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5207 00000100:00100000:0.0:1789760712.867187:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696992886087 time 697105386334 state 0x2 ref 0 00000100:00100000:3.0:1789760712.874791:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760712.874865:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495740928 00000100:00100000:0.0:1789760712.874875:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495740928 00000100:00100000:0.0:1789760712.874929:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 696992886087 time 697105386334 state 0x0 ref 1 00000100:00100000:0.0:1789760712.874945:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697105646750 time 697105386334 state 0x1 ref 1 00000100:00100000:0.0:1789760712.874975:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697105646750 time 697195691923 state 0x0 ref 1 00000100:00100000:0.0:1789760712.874990:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fdc9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495740928:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760712.875036:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21213184, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:0.0:1789760712.875045:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545569 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.875060:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760712.875065:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760712.875071:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545569 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.875084:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545569 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760712.936909:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21213184 -> 21217280 00010000:00010000:1.0:1789760712.936933:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545569 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.936947:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545569 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.936957:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545569 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.936965:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760712.936971:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545569 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.936981:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545569 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.937004:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545569 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.937018:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545569 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760712.937123:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fdc9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495740928:12345-192.168.203.9@tcp:4:dd.0 Request processed in 62129us (62335us total) trans 4294972496 rc 0/0 00000100:00100000:1.0:1789760712.937134:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5208 00000100:00100000:1.0:1789760712.937145:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697105646750 time 697195691923 state 0x2 ref 0 00000001:00080000:1.0:1789760712.937197:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972496 is committed 00000100:00100000:2.0:1789760712.945589:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760712.946600:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495741056 00000100:00100000:3.0:1789760712.946611:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495741056 00000100:00100000:3.0:1789760712.946669:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697105646750 time 697195691923 state 0x0 ref 1 00000100:00100000:3.0:1789760712.946684:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697195952339 time 697195691923 state 0x1 ref 1 00000100:00100000:3.0:1789760712.946762:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697195952339 time 697267478297 state 0x0 ref 1 00000100:00100000:3.0:1789760712.946775:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495741056:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760712.946824:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21217280, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:3.0:1789760712.946834:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545570 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.946850:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:3.0:1789760712.946855:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760712.946861:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545570 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.946881:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545570 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760713.069873:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21217280 -> 21221376 00010000:00010000:1.0:1789760713.069894:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545570 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.069907:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545570 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.069917:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545570 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.069925:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760713.069929:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545570 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.069939:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545570 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.069964:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545570 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.069973:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545570 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.070066:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495741056:12345-192.168.203.9@tcp:4:dd.0 Request processed in 123288us (124484us total) trans 4294972497 rc 0/0 00000100:00100000:1.0:1789760713.070077:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5209 00000100:00100000:1.0:1789760713.070086:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697195952339 time 697267478297 state 0x2 ref 0 00000001:00080000:1.0:1789760713.070133:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972497 is committed 00000100:00100000:2.0:1789760713.075628:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760713.075753:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495741184 00000100:00100000:0.0:1789760713.075762:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495741184 00000100:00100000:0.0:1789760713.075830:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697195952339 time 697267478297 state 0x0 ref 1 00000100:00100000:0.0:1789760713.075844:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697267738713 time 697267478297 state 0x1 ref 1 00000100:00100000:0.0:1789760713.075872:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697267738713 time 697396589168 state 0x0 ref 1 00000100:00100000:0.0:1789760713.075886:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495741184:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760713.075931:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21221376, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:0.0:1789760713.075940:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545577 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.075962:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760713.075966:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760713.075972:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545577 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.075984:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545577 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760713.079808:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760713.127953:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972498 is committed 00002000:00010000:0.0:1789760713.128945:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21221376 -> 21225472 00010000:00010000:0.0:1789760713.128964:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545577 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.128976:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545577 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.128984:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545577 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.128991:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760713.128997:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545577 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.129005:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545577 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.129024:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545577 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.129032:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545577 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760713.129131:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495741184:12345-192.168.203.9@tcp:4:dd.0 Request processed in 53243us (53507us total) trans 4294972498 rc 0/0 00000100:00100000:0.0:1789760713.129141:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5210 00000100:00100000:0.0:1789760713.129151:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697267738713 time 697396589168 state 0x2 ref 0 00000100:00100000:2.0:1789760713.137058:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760713.143330:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495741312 00000100:00100000:1.0:1789760713.143345:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495741312 00000100:00100000:1.0:1789760713.143413:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697267738713 time 697396589168 state 0x0 ref 1 00000100:00100000:1.0:1789760713.143432:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697396849584 time 697396589168 state 0x1 ref 1 00000100:00100000:1.0:1789760713.143734:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697396849584 time 697464448935 state 0x0 ref 1 00000100:00100000:1.0:1789760713.143750:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495741312:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760713.143805:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21225472, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:1.0:1789760713.143815:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454557e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.143838:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:1.0:1789760713.143843:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.143849:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454557e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.143864:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454557e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760713.180462:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21225472 -> 21229568 00010000:00010000:1.0:1789760713.180483:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454557e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.180497:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454557e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.180506:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454557e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.180513:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760713.180518:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454557e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.180527:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454557e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.180547:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454557e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.180555:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454557e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.180919:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495741312:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36893us (43590us total) trans 4294972499 rc 0/0 00000100:00100000:1.0:1789760713.180932:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5211 00000100:00100000:1.0:1789760713.180943:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697396849584 time 697464448935 state 0x2 ref 0 00000001:00080000:1.0:1789760713.180982:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972499 is committed 00000100:00100000:0.0:1789760713.184847:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760713.184936:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495741440 00000100:00100000:1.0:1789760713.184945:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495741440 00000100:00100000:1.0:1789760713.185001:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697396849584 time 697464448935 state 0x0 ref 1 00000100:00100000:1.0:1789760713.185017:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697464709351 time 697464448935 state 0x1 ref 1 00000100:00100000:1.0:1789760713.185050:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697464709351 time 697505766726 state 0x0 ref 1 00000100:00100000:1.0:1789760713.185062:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903639c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495741440:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760713.185110:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21229568, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:1.0:1789760713.185120:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545585 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.185135:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:1.0:1789760713.185140:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.185147:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545585 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.185161:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545585 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760713.206203:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760713.206262:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760713.206336:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760713.209951:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495741696 00000100:00100000:3.0:1789760713.209965:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495741696 00000100:00100000:3.0:1789760713.210046:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495741824 00000100:00100000:3.0:1789760713.210050:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495741824 00000100:00100000:3.0:1789760713.210078:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 455 00000100:00100000:3.0:1789760713.210088:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f871500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:17022965-1e4f-46fb-915c-61730096b8e2+8:2357:x1876699495741696:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:3.0:1789760713.210171:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f871500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:17022965-1e4f-46fb-915c-61730096b8e2+8:2357:x1876699495741696:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 85us (3978us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760713.210179:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 455 00000100:00100000:3.0:1789760713.210199:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 456 00000100:00100000:3.0:1789760713.210206:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f871f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2357:x1876699495741824:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:3.0:1789760713.210243:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f871f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2357:x1876699495741824:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 26us (3895us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760713.210250:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 456 00000100:00100000:3.0:1789760713.210305:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495741568 00000100:00100000:3.0:1789760713.210310:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495741568 00000100:00100000:3.0:1789760713.210335:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 390 00000100:00100000:3.0:1789760713.210341:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929012314050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2357:x1876699495741568:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:3.0:1789760713.210366:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929012314050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2357:x1876699495741568:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 25us (4112us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760713.210372:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 390 00000100:00080000:0.0:1789760713.211143:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:3.0:1789760713.249835:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972500 is committed 00002000:00010000:1.0:1789760713.249920:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21229568 -> 21233664 00010000:00010000:1.0:1789760713.249936:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545585 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.249949:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545585 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.249958:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545585 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.249965:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760713.249969:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545585 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.249977:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545585 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.249996:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545585 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.250003:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545585 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.250138:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903639c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495741440:12345-192.168.203.9@tcp:4:dd.0 Request processed in 65073us (65294us total) trans 4294972500 rc 0/0 00000100:00100000:1.0:1789760713.250149:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5212 00000100:00100000:1.0:1789760713.250159:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697464709351 time 697505766726 state 0x2 ref 0 00000100:00100000:0.0:1789760713.258066:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760713.258167:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495741952 00000100:00100000:2.0:1789760713.258177:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495741952 00000100:00100000:2.0:1789760713.260553:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697464709351 time 697505766726 state 0x0 ref 1 00000100:00100000:2.0:1789760713.260599:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697506027142 time 697505766726 state 0x1 ref 1 00000100:00100000:2.0:1789760713.260650:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697506027142 time 697581358639 state 0x0 ref 1 00000100:00100000:2.0:1789760713.260675:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f873480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495741952:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760713.260725:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21233664, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:2.0:1789760713.260735:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454558c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.260751:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:2.0:1789760713.260971:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760713.260979:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454558c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.260995:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454558c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760713.294649:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972501 is committed 00002000:00010000:3.0:1789760713.295104:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21233664 -> 21237760 00010000:00010000:3.0:1789760713.295121:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454558c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.295134:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454558c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.295143:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454558c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.295150:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760713.295155:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454558c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.295163:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454558c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.295184:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454558c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.295192:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454558c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760713.295345:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f873480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495741952:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34680us (37286us total) trans 4294972501 rc 0/0 00000100:00100000:3.0:1789760713.295365:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5213 00000100:00100000:3.0:1789760713.295376:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697506027142 time 697581358639 state 0x2 ref 0 00000100:00100000:0.0:1789760713.299109:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760713.299192:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495742080 00000100:00100000:1.0:1789760713.299201:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495742080 00000100:00100000:1.0:1789760713.299271:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697506027142 time 697581358639 state 0x0 ref 1 00000100:00100000:1.0:1789760713.299328:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697581619055 time 697581358639 state 0x1 ref 1 00000100:00100000:1.0:1789760713.299359:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697581619055 time 697620071225 state 0x0 ref 1 00000100:00100000:1.0:1789760713.299371:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f921c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495742080:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760713.299419:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21237760, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:1.0:1789760713.299428:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545593 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.299445:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:1.0:1789760713.299449:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.299456:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545593 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.299471:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545593 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760713.342161:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21237760 -> 21241856 00010000:00010000:1.0:1789760713.342182:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545593 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.342195:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545593 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.342204:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545593 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.342212:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760713.342217:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545593 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.342225:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545593 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.342265:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545593 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.342273:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545593 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.342442:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f921c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495742080:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43068us (43339us total) trans 4294972502 rc 0/0 00000100:00100000:1.0:1789760713.342457:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5214 00000100:00100000:1.0:1789760713.342467:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697581619055 time 697620071225 state 0x2 ref 0 00000001:00080000:1.0:1789760713.342506:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972502 is committed 00000100:00100000:0.0:1789760713.346965:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760713.347043:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495742208 00000100:00100000:1.0:1789760713.347053:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495742208 00000100:00100000:1.0:1789760713.347103:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697581619055 time 697620071225 state 0x0 ref 1 00000100:00100000:1.0:1789760713.347119:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697620331641 time 697620071225 state 0x1 ref 1 00000100:00100000:1.0:1789760713.347148:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697620331641 time 697667865361 state 0x0 ref 1 00000100:00100000:1.0:1789760713.347160:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f923100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495742208:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760713.347204:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21241856, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:1.0:1789760713.347213:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454559a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.347238:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:1.0:1789760713.347243:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.347250:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454559a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.347271:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454559a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760713.420748:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21241856 -> 21245952 00010000:00010000:1.0:1789760713.420768:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454559a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.420782:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454559a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.420791:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454559a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.420799:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760713.420804:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454559a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.420815:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454559a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.420834:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454559a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.420842:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454559a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.420933:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f923100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495742208:12345-192.168.203.9@tcp:4:dd.0 Request processed in 73770us (73972us total) trans 4294972503 rc 0/0 00000100:00100000:1.0:1789760713.420943:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5215 00000100:00100000:1.0:1789760713.420953:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697620331641 time 697667865361 state 0x2 ref 0 00000001:00080000:1.0:1789760713.420998:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972503 is committed 00000100:00100000:0.0:1789760713.425905:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760713.425997:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495742336 00000100:00100000:3.0:1789760713.426009:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495742336 00000100:00100000:3.0:1789760713.426065:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697620331641 time 697667865361 state 0x0 ref 1 00000100:00100000:3.0:1789760713.426081:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697668125777 time 697667865361 state 0x1 ref 1 00000100:00100000:3.0:1789760713.426114:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697668125777 time 697746830777 state 0x0 ref 1 00000100:00100000:3.0:1789760713.426127:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f922300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495742336:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760713.426195:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21245952, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:3.0:1789760713.426206:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545455a1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.426223:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:3.0:1789760713.426228:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.426488:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545455a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.426506:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545455a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760713.471790:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972504 is committed 00002000:00010000:0.0:1789760713.472005:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21245952 -> 21250048 00010000:00010000:0.0:1789760713.472021:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545455a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.472034:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545455a1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.472044:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545455a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.472057:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760713.472062:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545455a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.472071:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545455a1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.472090:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545455a1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.472098:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545455a1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760713.472352:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f922300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495742336:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46222us (46453us total) trans 4294972504 rc 0/0 00000100:00100000:0.0:1789760713.472361:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5216 00000100:00100000:0.0:1789760713.472371:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697668125777 time 697746830777 state 0x2 ref 0 00000100:00100000:0.0:1789760713.482410:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760713.482482:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495742464 00000100:00100000:1.0:1789760713.482492:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495742464 00000100:00100000:1.0:1789760713.482543:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697668125777 time 697746830777 state 0x0 ref 1 00000100:00100000:1.0:1789760713.482558:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697747091193 time 697746830777 state 0x1 ref 1 00000100:00100000:1.0:1789760713.482596:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697747091193 time 697803312989 state 0x0 ref 1 00000100:00100000:1.0:1789760713.482608:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495742464:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760713.482654:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21250048, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:1.0:1789760713.482663:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545455a8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.482680:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:1.0:1789760713.482684:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.482691:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545455a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.482706:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545455a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760713.523039:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972505 is committed 00002000:00010000:1.0:1789760713.523875:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21250048 -> 21254144 00010000:00010000:1.0:1789760713.523900:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545455a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.523920:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545455a8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.523930:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545455a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.523937:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760713.523942:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545455a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.523953:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545455a8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.523972:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545455a8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.523980:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545455a8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.524075:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495742464:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41463us (41670us total) trans 4294972505 rc 0/0 00000100:00100000:1.0:1789760713.524085:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5217 00000100:00100000:1.0:1789760713.524095:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697747091193 time 697803312989 state 0x2 ref 0 00000100:00100000:0.0:1789760713.527821:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760713.527966:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495742592 00000100:00100000:1.0:1789760713.527975:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495742592 00000100:00100000:1.0:1789760713.528026:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697747091193 time 697803312989 state 0x0 ref 1 00000100:00100000:1.0:1789760713.528040:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697803573405 time 697803312989 state 0x1 ref 1 00000100:00100000:1.0:1789760713.528108:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697803573405 time 697848823362 state 0x0 ref 1 00000100:00100000:1.0:1789760713.528120:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495742592:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760713.528164:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21254144, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:1.0:1789760713.528173:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545455af lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.528189:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:1.0:1789760713.528193:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.528200:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545455af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.528214:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545455af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.567438:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760713.646009:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972506 is committed 00002000:00010000:1.0:1789760713.646329:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21254144 -> 21258240 00010000:00010000:1.0:1789760713.646345:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545455af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.646359:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545455af lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.646373:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545455af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.646381:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760713.646386:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545455af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.646395:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545455af lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.646414:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545455af lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.646422:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545455af lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760713.646829:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495742592:12345-192.168.203.9@tcp:4:dd.0 Request processed in 118705us (119010us total) trans 4294972506 rc 0/0 00000100:00100000:2.0:1789760713.646841:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5218 00000100:00100000:2.0:1789760713.646851:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697803573405 time 697848823362 state 0x2 ref 0 00000100:00100000:0.0:1789760713.665871:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760713.665974:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495742720 00000100:00100000:2.0:1789760713.665984:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495742720 00000100:00100000:2.0:1789760713.666039:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697803573405 time 697848823362 state 0x0 ref 1 00000100:00100000:2.0:1789760713.666064:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697849083778 time 697848823362 state 0x1 ref 1 00000100:00100000:2.0:1789760713.666092:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697849083778 time 697986809150 state 0x0 ref 1 00000100:00100000:2.0:1789760713.666104:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495742720:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760713.666149:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21258240, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:2.0:1789760713.666157:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545455b6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.666173:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:2.0:1789760713.666177:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760713.666185:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545455b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.666197:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545455b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760713.706041:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972507 is committed 00002000:00010000:1.0:1789760713.706223:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21258240 -> 21262336 00002000:00010000:1.0:1789760713.706228:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760712 -> 1789760713 00002000:00010000:1.0:1789760713.706230:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760712 -> 1789760713 00010000:00010000:1.0:1789760713.706244:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545455b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.706897:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545455b6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.706912:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545455b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.706918:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760713.706923:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545455b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.706931:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545455b6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.706952:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545455b6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.706960:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545455b6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.707047:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495742720:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40941us (41181us total) trans 4294972507 rc 0/0 00000100:00100000:1.0:1789760713.707059:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5219 00000100:00100000:1.0:1789760713.707070:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697849083778 time 697986809150 state 0x2 ref 0 00000100:00100000:0.0:1789760713.710958:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760713.711038:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495742848 00000100:00100000:1.0:1789760713.711049:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495742848 00000100:00100000:1.0:1789760713.711105:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697849083778 time 697986809150 state 0x0 ref 1 00000100:00100000:1.0:1789760713.711121:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697987069566 time 697986809150 state 0x1 ref 1 00000100:00100000:1.0:1789760713.711152:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697987069566 time 698031869650 state 0x0 ref 1 00000100:00100000:1.0:1789760713.711164:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495742848:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760713.711209:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21262336, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:1.0:1789760713.711217:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545455bd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.711243:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:1.0:1789760713.711255:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.711262:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545455bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.711276:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545455bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760713.748335:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972508 is committed 00002000:00010000:1.0:1789760713.748616:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21262336 -> 21266432 00010000:00010000:1.0:1789760713.748632:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545455bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.748648:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545455bd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.748657:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545455bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.748664:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760713.748670:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545455bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.748680:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545455bd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.748699:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545455bd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.748707:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545455bd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.748804:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495742848:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37636us (37850us total) trans 4294972508 rc 0/0 00000100:00100000:1.0:1789760713.748814:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5220 00000100:00100000:1.0:1789760713.748824:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697987069566 time 698031869650 state 0x2 ref 0 00000100:00100000:0.0:1789760713.752638:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760713.752703:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495742976 00000100:00100000:2.0:1789760713.752714:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495742976 00000100:00100000:2.0:1789760713.752836:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 697987069566 time 698031869650 state 0x0 ref 1 00000100:00100000:2.0:1789760713.752854:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698032130066 time 698031869650 state 0x1 ref 1 00000100:00100000:2.0:1789760713.752884:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698032130066 time 698073600338 state 0x0 ref 1 00000100:00100000:2.0:1789760713.752898:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495742976:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760713.752945:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21266432, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:2.0:1789760713.752954:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545455c4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.752980:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:2.0:1789760713.752986:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760713.752993:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545455c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.753006:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545455c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760713.782373:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972509 is committed 00002000:00010000:0.0:1789760713.782480:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21266432 -> 21270528 00010000:00010000:0.0:1789760713.782496:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545455c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.782518:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545455c4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.782533:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545455c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.782540:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760713.782545:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545455c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.782552:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545455c4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.782572:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545455c4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.782579:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545455c4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760713.782688:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495742976:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29789us (30055us total) trans 4294972509 rc 0/0 00000100:00100000:0.0:1789760713.782697:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5221 00000100:00100000:0.0:1789760713.782707:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698032130066 time 698073600338 state 0x2 ref 0 00000100:00100000:0.0:1789760713.786580:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760713.786898:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495743104 00000100:00100000:1.0:1789760713.786907:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495743104 00000100:00100000:1.0:1789760713.786960:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698032130066 time 698073600338 state 0x0 ref 1 00000100:00100000:1.0:1789760713.786973:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698073860754 time 698073600338 state 0x1 ref 1 00000100:00100000:1.0:1789760713.787000:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698073860754 time 698107717025 state 0x0 ref 1 00000100:00100000:1.0:1789760713.787011:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495743104:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760713.787059:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21270528, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:1.0:1789760713.787068:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545455cb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.787083:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:1.0:1789760713.787088:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.787094:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545455cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.787108:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545455cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760713.829375:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972510 is committed 00002000:00010000:2.0:1789760713.829481:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21270528 -> 21274624 00010000:00010000:2.0:1789760713.829497:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545455cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.829509:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545455cb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.829524:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545455cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.829532:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760713.829536:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545455cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.829545:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545455cb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.829564:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545455cb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.829584:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545455cb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760713.829674:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495743104:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42660us (43110us total) trans 4294972510 rc 0/0 00000100:00100000:2.0:1789760713.829685:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5222 00000100:00100000:2.0:1789760713.829695:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698073860754 time 698107717025 state 0x2 ref 0 00000100:00100000:1.0:1789760713.833012:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760713.833087:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495743232 00000100:00100000:0.0:1789760713.833097:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495743232 00000100:00100000:0.0:1789760713.833147:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698073860754 time 698107717025 state 0x0 ref 1 00000100:00100000:0.0:1789760713.833162:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698107977441 time 698107717025 state 0x1 ref 1 00000100:00100000:0.0:1789760713.833189:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698107977441 time 698153905537 state 0x0 ref 1 00000100:00100000:0.0:1789760713.833199:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495743232:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760713.833366:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21274624, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:0.0:1789760713.833376:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545455d2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.833391:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760713.833395:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760713.833401:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545455d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.833414:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545455d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760713.858125:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972511 is committed 00002000:00010000:0.0:1789760713.858239:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21274624 -> 21278720 00010000:00010000:0.0:1789760713.858272:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545455d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.858423:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545455d2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.858432:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545455d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.858440:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760713.858445:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545455d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.858455:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545455d2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.858474:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545455d2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.858482:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545455d2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760713.861497:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495743232:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28284us (28480us total) trans 4294972511 rc 0/0 00000100:00100000:0.0:1789760713.861514:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5223 00000100:00100000:0.0:1789760713.861528:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698107977441 time 698153905537 state 0x2 ref 0 00000100:00100000:0.0:1789760713.863182:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760713.863413:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495743360 00000100:00100000:2.0:1789760713.863422:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495743360 00000100:00100000:2.0:1789760713.863469:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698107977441 time 698153905537 state 0x0 ref 1 00000100:00100000:2.0:1789760713.863483:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698154165953 time 698153905537 state 0x1 ref 1 00000100:00100000:2.0:1789760713.863544:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698154165953 time 698184260094 state 0x0 ref 1 00000100:00100000:2.0:1789760713.863556:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495743360:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760713.863612:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21278720, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:2.0:1789760713.863621:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545455d9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.863638:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:2.0:1789760713.863643:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760713.863649:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545455d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.863664:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545455d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760713.897110:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21278720 -> 21282816 00010000:00010000:2.0:1789760713.897135:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545455d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.897151:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545455d9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.897161:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545455d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.897169:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760713.897175:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545455d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.897184:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545455d9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.897205:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545455d9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.897213:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545455d9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760713.897364:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495743360:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33803us (34184us total) trans 4294972512 rc 0/0 00000100:00100000:2.0:1789760713.897375:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5224 00000100:00100000:2.0:1789760713.897385:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698154165953 time 698184260094 state 0x2 ref 0 00000001:00080000:1.0:1789760713.899017:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972512 is committed 00000100:00100000:2.0:1789760713.900681:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760713.900745:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495743488 00000100:00100000:1.0:1789760713.900762:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495743488 00000100:00100000:1.0:1789760713.900903:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698154165953 time 698184260094 state 0x0 ref 1 00000100:00100000:1.0:1789760713.900915:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698184520510 time 698184260094 state 0x1 ref 1 00000100:00100000:1.0:1789760713.900939:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698184520510 time 698221655485 state 0x0 ref 1 00000100:00100000:1.0:1789760713.900951:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495743488:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760713.901018:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21282816, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:1.0:1789760713.901029:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545455e0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.901045:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:1.0:1789760713.901054:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.901061:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545455e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.901076:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545455e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760713.937073:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972513 is committed 00002000:00010000:1.0:1789760713.937270:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21282816 -> 21286912 00010000:00010000:1.0:1789760713.937303:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545455e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.937316:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545455e0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.937325:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545455e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.937333:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760713.937338:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545455e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.937347:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545455e0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.937366:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545455e0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.937386:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545455e0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.937479:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495743488:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36524us (36799us total) trans 4294972513 rc 0/0 00000100:00100000:1.0:1789760713.937490:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5225 00000100:00100000:1.0:1789760713.937501:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698184520510 time 698221655485 state 0x2 ref 0 00000100:00100000:1.0:1789760713.942368:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760713.942511:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495743616 00000100:00100000:0.0:1789760713.942520:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495743616 00000100:00100000:0.0:1789760713.942629:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698184520510 time 698221655485 state 0x0 ref 1 00000100:00100000:0.0:1789760713.942646:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698221915901 time 698221655485 state 0x1 ref 1 00000100:00100000:0.0:1789760713.942702:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698221915901 time 698263418018 state 0x0 ref 1 00000100:00100000:0.0:1789760713.942714:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495743616:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760713.942757:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21286912, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:0.0:1789760713.942767:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545455e7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.942782:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760713.942786:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760713.942792:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545455e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.942805:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545455e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760713.970709:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972514 is committed 00002000:00010000:1.0:1789760713.970824:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21286912 -> 21291008 00010000:00010000:1.0:1789760713.970839:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545455e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.970852:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545455e7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.970862:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545455e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.970869:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760713.970875:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545455e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.970884:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545455e7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.970909:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545455e7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.970917:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545455e7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.971450:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495743616:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28732us (29084us total) trans 4294972514 rc 0/0 00000100:00100000:1.0:1789760713.971462:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5226 00000100:00100000:1.0:1789760713.971473:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698221915901 time 698263418018 state 0x2 ref 0 00000100:00100000:0.0:1789760713.978317:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760713.978477:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495743744 00000100:00100000:2.0:1789760713.978486:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495743744 00000100:00100000:2.0:1789760713.978538:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698221915901 time 698263418018 state 0x0 ref 1 00000100:00100000:2.0:1789760713.978553:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698263678434 time 698263418018 state 0x1 ref 1 00000100:00100000:2.0:1789760713.978682:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698263678434 time 698299398452 state 0x0 ref 1 00000100:00100000:2.0:1789760713.978696:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495743744:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760713.978742:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21291008, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:2.0:1789760713.978752:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545455ee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.978768:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:2.0:1789760713.978772:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760713.978779:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545455ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.978793:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545455ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760714.014207:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21291008 -> 21295104 00010000:00010000:3.0:1789760714.014227:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545455ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.014263:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545455ee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.014271:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545455ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.014399:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760714.014405:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545455ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.014415:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545455ee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.014434:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545455ee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.014441:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545455ee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.014661:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495743744:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35962us (36348us total) trans 4294972515 rc 0/0 00000100:00100000:3.0:1789760714.014673:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5227 00000100:00100000:3.0:1789760714.014684:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698263678434 time 698299398452 state 0x2 ref 0 00000001:00080000:1.0:1789760714.015445:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972515 is committed 00000100:00100000:2.0:1789760714.020165:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760714.020314:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495743872 00000100:00100000:1.0:1789760714.020325:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495743872 00000100:00100000:1.0:1789760714.020379:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698263678434 time 698299398452 state 0x0 ref 1 00000100:00100000:1.0:1789760714.020395:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698299658868 time 698299398452 state 0x1 ref 1 00000100:00100000:1.0:1789760714.020426:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698299658868 time 698341142675 state 0x0 ref 1 00000100:00100000:1.0:1789760714.020438:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495743872:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760714.020484:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21295104, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:1.0:1789760714.020494:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545455f5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.020518:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:1.0:1789760714.020522:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.020527:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545455f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.020541:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545455f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760714.084070:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972516 is committed 00002000:00010000:3.0:1789760714.084167:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21295104 -> 21299200 00010000:00010000:3.0:1789760714.084182:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545455f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.084195:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545455f5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.084204:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545455f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.084211:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760714.084216:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545455f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.084240:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545455f5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.084267:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545455f5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.084275:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545455f5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.084405:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495743872:12345-192.168.203.9@tcp:4:dd.0 Request processed in 63964us (64242us total) trans 4294972516 rc 0/0 00000100:00100000:3.0:1789760714.084415:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5228 00000100:00100000:3.0:1789760714.084425:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698299658868 time 698341142675 state 0x2 ref 0 00000100:00100000:2.0:1789760714.091972:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760714.092076:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495744000 00000100:00100000:1.0:1789760714.092086:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495744000 00000100:00100000:1.0:1789760714.092139:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698299658868 time 698341142675 state 0x0 ref 1 00000100:00100000:1.0:1789760714.092154:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698341403091 time 698341142675 state 0x1 ref 1 00000100:00100000:1.0:1789760714.092185:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698341403091 time 698412901172 state 0x0 ref 1 00000100:00100000:1.0:1789760714.092197:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495744000:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760714.092261:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21299200, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:1.0:1789760714.092270:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545455fc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.092324:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:1.0:1789760714.092329:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.092335:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545455fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.092354:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545455fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760714.158799:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21299200 -> 21303296 00010000:00010000:2.0:1789760714.159122:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545455fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.159135:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545455fc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.159144:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545455fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.159151:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760714.159155:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545455fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.159164:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545455fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.159183:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545455fc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.159190:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545455fc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760714.159506:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972517 is committed 00000100:00100000:2.0:1789760714.160008:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495744000:12345-192.168.203.9@tcp:4:dd.0 Request processed in 67807us (68039us total) trans 4294972517 rc 0/0 00000100:00100000:2.0:1789760714.160020:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5229 00000100:00100000:2.0:1789760714.160031:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698341403091 time 698412901172 state 0x2 ref 0 00000100:00100000:1.0:1789760714.171978:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760714.172082:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495744128 00000100:00100000:3.0:1789760714.172095:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495744128 00000100:00100000:3.0:1789760714.172150:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698341403091 time 698412901172 state 0x0 ref 1 00000100:00100000:3.0:1789760714.172165:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698413161588 time 698412901172 state 0x1 ref 1 00000100:00100000:3.0:1789760714.172200:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698413161588 time 698492916966 state 0x0 ref 1 00000100:00100000:3.0:1789760714.172213:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495744128:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760714.172271:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21303296, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:3.0:1789760714.172319:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545603 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.172336:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760714.172342:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.172348:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545603 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.172363:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545603 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760714.216731:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21303296 -> 21307392 00010000:00010000:3.0:1789760714.216754:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545603 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.216772:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545603 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.216781:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545603 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.216789:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760714.216794:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545603 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.216803:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545603 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.216825:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545603 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.216832:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545603 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.216929:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495744128:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44713us (44955us total) trans 4294972518 rc 0/0 00000100:00100000:3.0:1789760714.216940:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5230 00000100:00100000:3.0:1789760714.216950:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698413161588 time 698492916966 state 0x2 ref 0 00000001:00080000:3.0:1789760714.216997:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972518 is committed 00000100:00100000:0.0:1789760714.220204:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760714.220791:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495744256 00000100:00100000:3.0:1789760714.220802:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495744256 00000100:00100000:3.0:1789760714.220861:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698413161588 time 698492916966 state 0x0 ref 1 00000100:00100000:3.0:1789760714.220876:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698493177382 time 698492916966 state 0x1 ref 1 00000100:00100000:3.0:1789760714.220913:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698493177382 time 698541629663 state 0x0 ref 1 00000100:00100000:3.0:1789760714.220926:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495744256:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760714.220985:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21307392, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:3.0:1789760714.220994:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454560a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.221012:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760714.221016:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.221023:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454560a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.221038:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454560a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760714.354139:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972519 is committed 00002000:00010000:1.0:1789760714.354307:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21307392 -> 21311488 00010000:00010000:1.0:1789760714.354322:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454560a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.354335:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454560a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.354343:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454560a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.354350:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760714.354356:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454560a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.354364:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454560a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.354383:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454560a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.354390:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454560a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.354483:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495744256:12345-192.168.203.9@tcp:4:dd.0 Request processed in 133554us (134283us total) trans 4294972519 rc 0/0 00000100:00100000:1.0:1789760714.354492:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5231 00000100:00100000:1.0:1789760714.354502:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698493177382 time 698541629663 state 0x2 ref 0 00000100:00100000:0.0:1789760714.363613:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760714.363779:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495744384 00000100:00100000:3.0:1789760714.363788:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495744384 00000100:00100000:3.0:1789760714.363841:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698493177382 time 698541629663 state 0x0 ref 1 00000100:00100000:3.0:1789760714.363857:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698541890079 time 698541629663 state 0x1 ref 1 00000100:00100000:3.0:1789760714.363888:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698541890079 time 698684605231 state 0x0 ref 1 00000100:00100000:3.0:1789760714.363901:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495744384:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760714.363949:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21311488, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:3.0:1789760714.363959:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545611 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.363975:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760714.363980:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.363987:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545611 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.364000:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545611 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760714.428842:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21311488 -> 21315584 00010000:00010000:3.0:1789760714.428860:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545611 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.428872:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545611 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.428881:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545611 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.428889:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760714.428895:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545611 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.428903:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545611 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.428923:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545611 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.428930:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545611 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.429020:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495744384:12345-192.168.203.9@tcp:4:dd.0 Request processed in 65116us (65414us total) trans 4294972520 rc 0/0 00000100:00100000:3.0:1789760714.429031:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5232 00000100:00100000:3.0:1789760714.429040:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698541890079 time 698684605231 state 0x2 ref 0 00000001:00080000:0.0:1789760714.432635:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972520 is committed 00000100:00100000:3.0:1789760714.433006:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760714.433310:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495744512 00000100:00100000:0.0:1789760714.433320:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495744512 00000100:00100000:0.0:1789760714.433375:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698541890079 time 698684605231 state 0x0 ref 1 00000100:00100000:0.0:1789760714.433390:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698684865647 time 698684605231 state 0x1 ref 1 00000100:00100000:0.0:1789760714.433418:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698684865647 time 698754134731 state 0x0 ref 1 00000100:00100000:0.0:1789760714.433432:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fdcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495744512:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760714.433554:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21315584, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:0.0:1789760714.433738:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545618 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.433755:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760714.433760:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760714.433766:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545618 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.433781:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545618 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760714.465923:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21315584 -> 21319680 00010000:00010000:0.0:1789760714.465945:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545618 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.465957:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545618 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.465970:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545618 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.465978:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760714.465984:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545618 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.465992:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545618 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.466011:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545618 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.466018:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545618 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760714.466108:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fdcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495744512:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32674us (33102us total) trans 4294972521 rc 0/0 00000100:00100000:0.0:1789760714.466119:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5233 00000100:00100000:0.0:1789760714.466128:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698684865647 time 698754134731 state 0x2 ref 0 00000001:00080000:1.0:1789760714.467559:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972521 is committed 00000100:00100000:1.0:1789760714.470996:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760714.471082:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495744640 00000100:00100000:3.0:1789760714.471092:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495744640 00000100:00100000:3.0:1789760714.471145:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698684865647 time 698754134731 state 0x0 ref 1 00000100:00100000:3.0:1789760714.471160:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698754395147 time 698754134731 state 0x1 ref 1 00000100:00100000:3.0:1789760714.471187:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698754395147 time 698791903143 state 0x0 ref 1 00000100:00100000:3.0:1789760714.471199:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495744640:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760714.471269:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21319680, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:3.0:1789760714.471405:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454561f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.471423:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760714.471428:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.471435:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454561f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.471450:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454561f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760714.509144:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21319680 -> 21323776 00010000:00010000:3.0:1789760714.509165:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454561f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.509179:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454561f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.509188:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454561f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.509195:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760714.509199:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454561f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.509208:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454561f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.509237:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454561f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.509254:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454561f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.509380:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495744640:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38177us (38387us total) trans 4294972522 rc 0/0 00000100:00100000:3.0:1789760714.509391:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5234 00000100:00100000:3.0:1789760714.509400:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698754395147 time 698791903143 state 0x2 ref 0 00000001:00080000:0.0:1789760714.511962:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972522 is committed 00000100:00100000:3.0:1789760714.513329:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760714.513423:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495744768 00000100:00100000:0.0:1789760714.513433:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495744768 00000100:00100000:0.0:1789760714.513498:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698754395147 time 698791903143 state 0x0 ref 1 00000100:00100000:0.0:1789760714.513518:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698792163559 time 698791903143 state 0x1 ref 1 00000100:00100000:0.0:1789760714.513548:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698792163559 time 698834264563 state 0x0 ref 1 00000100:00100000:0.0:1789760714.513561:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fdc9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495744768:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760714.513696:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21323776, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:0.0:1789760714.513706:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545626 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.513720:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760714.513724:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760714.513730:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545626 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.513743:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545626 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760714.545575:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972523 is committed 00002000:00010000:3.0:1789760714.545787:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21323776 -> 21327872 00010000:00010000:3.0:1789760714.545804:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545626 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.545819:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545626 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.545829:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545626 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.545835:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760714.545840:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545626 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.545852:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545626 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.545871:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545626 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.545878:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545626 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.545967:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fdc9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495744768:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32403us (32643us total) trans 4294972523 rc 0/0 00000100:00100000:3.0:1789760714.545977:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5235 00000100:00100000:3.0:1789760714.545987:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698792163559 time 698834264563 state 0x2 ref 0 00000100:00100000:0.0:1789760714.550339:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760714.550444:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495744896 00000100:00100000:3.0:1789760714.550454:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495744896 00000100:00100000:3.0:1789760714.550506:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698792163559 time 698834264563 state 0x0 ref 1 00000100:00100000:3.0:1789760714.550521:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698834524979 time 698834264563 state 0x1 ref 1 00000100:00100000:3.0:1789760714.550543:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698834524979 time 698871260271 state 0x0 ref 1 00000100:00100000:3.0:1789760714.550557:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495744896:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760714.550612:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21327872, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:3.0:1789760714.550629:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454562d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.550646:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760714.550650:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.550656:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454562d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.550670:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454562d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760714.583978:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972524 is committed 00002000:00010000:3.0:1789760714.584140:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21327872 -> 21331968 00010000:00010000:3.0:1789760714.584154:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454562d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.584166:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454562d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.584175:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454562d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.584181:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760714.584186:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454562d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.584194:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454562d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.584212:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454562d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.584324:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454562d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.584406:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495744896:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33848us (34073us total) trans 4294972524 rc 0/0 00000100:00100000:3.0:1789760714.584416:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5236 00000100:00100000:3.0:1789760714.584425:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698834524979 time 698871260271 state 0x2 ref 0 00000100:00100000:1.0:1789760714.588861:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760714.588948:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495745024 00000100:00100000:3.0:1789760714.588958:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495745024 00000100:00100000:3.0:1789760714.589013:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698834524979 time 698871260271 state 0x0 ref 1 00000100:00100000:3.0:1789760714.589028:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698871520687 time 698871260271 state 0x1 ref 1 00000100:00100000:3.0:1789760714.589099:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698871520687 time 698909814712 state 0x0 ref 1 00000100:00100000:3.0:1789760714.589111:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495745024:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760714.589161:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21331968, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:3.0:1789760714.589170:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545634 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.589184:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760714.589189:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.589195:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545634 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.589214:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545634 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.591555:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760714.628254:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972525 is committed 00002000:00010000:2.0:1789760714.628445:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21331968 -> 21336064 00010000:00010000:2.0:1789760714.628527:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545634 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.628543:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545634 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.628551:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545634 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.628682:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760714.628688:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545634 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.628697:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545634 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.628716:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545634 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.628724:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545634 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760714.628845:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495745024:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39731us (39988us total) trans 4294972525 rc 0/0 00000100:00100000:2.0:1789760714.628856:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5237 00000100:00100000:2.0:1789760714.628866:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698871520687 time 698909814712 state 0x2 ref 0 00000100:00100000:0.0:1789760714.634110:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760714.634207:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495745152 00000100:00100000:3.0:1789760714.634783:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495745152 00000100:00100000:3.0:1789760714.634841:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698871520687 time 698909814712 state 0x0 ref 1 00000100:00100000:3.0:1789760714.634856:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698910075128 time 698909814712 state 0x1 ref 1 00000100:00100000:3.0:1789760714.634889:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698910075128 time 698955605670 state 0x0 ref 1 00000100:00100000:3.0:1789760714.634902:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495745152:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760714.634947:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21336064, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:3.0:1789760714.634956:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454563b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.634971:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760714.634976:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.634983:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454563b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.634997:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454563b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760714.672857:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972526 is committed 00002000:00010000:3.0:1789760714.673001:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21336064 -> 21340160 00002000:00010000:3.0:1789760714.673005:0:6714:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760713 -> 1789760714 00002000:00010000:3.0:1789760714.673007:0:6714:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760713 -> 1789760714 00010000:00010000:3.0:1789760714.673020:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454563b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.673034:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454563b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.673042:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454563b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.673050:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760714.673055:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454563b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.673063:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454563b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.673083:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454563b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.673091:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454563b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.673229:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495745152:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38316us (39117us total) trans 4294972526 rc 0/0 00000100:00100000:3.0:1789760714.673248:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5238 00000100:00100000:3.0:1789760714.673258:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698910075128 time 698955605670 state 0x2 ref 0 00000100:00100000:0.0:1789760714.680763:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760714.681761:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495745280 00000100:00100000:3.0:1789760714.681771:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495745280 00000100:00100000:2.0:1789760714.681905:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698910075128 time 698955605670 state 0x0 ref 1 00000100:00100000:2.0:1789760714.682151:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698955866086 time 698955605670 state 0x1 ref 1 00000100:00100000:2.0:1789760714.682225:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698955866086 time 699002940768 state 0x0 ref 1 00000100:00100000:2.0:1789760714.682239:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495745280:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760714.682318:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21340160, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:2.0:1789760714.682328:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545642 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.682343:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:2.0:1789760714.682347:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.682353:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545642 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.682368:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545642 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760714.744965:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972527 is committed 00002000:00010000:2.0:1789760714.745045:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21340160 -> 21344256 00010000:00010000:2.0:1789760714.745061:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545642 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.745073:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545642 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.745082:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545642 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.745089:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760714.745093:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545642 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.745107:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545642 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.745126:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545642 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.745133:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545642 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760714.745234:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495745280:12345-192.168.203.9@tcp:4:dd.0 Request processed in 62982us (64464us total) trans 4294972527 rc 0/0 00000100:00100000:2.0:1789760714.745252:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5239 00000100:00100000:2.0:1789760714.745262:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698955866086 time 699002940768 state 0x2 ref 0 00000100:00100000:1.0:1789760714.754901:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760714.754995:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495745408 00000100:00100000:2.0:1789760714.755004:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495745408 00000100:00100000:2.0:1789760714.755062:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 698955866086 time 699002940768 state 0x0 ref 1 00000100:00100000:2.0:1789760714.755079:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699003201184 time 699002940768 state 0x1 ref 1 00000100:00100000:2.0:1789760714.755110:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699003201184 time 699075827083 state 0x0 ref 1 00000100:00100000:2.0:1789760714.755122:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495745408:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760714.755168:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21344256, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:2.0:1789760714.755176:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545649 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.755194:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:2.0:1789760714.755198:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.755205:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545649 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.755228:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545649 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760714.813717:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972528 is committed 00002000:00010000:2.0:1789760714.813975:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21344256 -> 21348352 00010000:00010000:2.0:1789760714.813991:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545649 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.814004:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545649 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.814013:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545649 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.814020:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760714.814025:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545649 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.814034:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545649 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.814054:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545649 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.814062:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545649 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760714.814150:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495745408:12345-192.168.203.9@tcp:4:dd.0 Request processed in 59025us (59254us total) trans 4294972528 rc 0/0 00000100:00100000:2.0:1789760714.814170:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5240 00000100:00100000:2.0:1789760714.814180:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699003201184 time 699075827083 state 0x2 ref 0 00000100:00100000:0.0:1789760714.824149:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760714.824476:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495745536 00000100:00100000:3.0:1789760714.824487:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495745536 00000100:00100000:3.0:1789760714.824540:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699003201184 time 699075827083 state 0x0 ref 1 00000100:00100000:3.0:1789760714.824555:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699076087499 time 699075827083 state 0x1 ref 1 00000100:00100000:3.0:1789760714.824587:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699076087499 time 699145304292 state 0x0 ref 1 00000100:00100000:3.0:1789760714.824599:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370bd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495745536:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760714.824643:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21348352, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:3.0:1789760714.824652:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545650 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.824668:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760714.824856:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.824863:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545650 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.824882:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545650 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760714.872508:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972529 is committed 00002000:00010000:2.0:1789760714.873384:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21348352 -> 21352448 00010000:00010000:2.0:1789760714.873401:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545650 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.873413:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545650 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.873421:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545650 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.873429:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760714.873433:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545650 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.873442:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545650 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.873477:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545650 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.873485:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545650 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760714.873593:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370bd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495745536:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48990us (49450us total) trans 4294972529 rc 0/0 00000100:00100000:2.0:1789760714.873604:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5241 00000100:00100000:2.0:1789760714.873614:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699076087499 time 699145304292 state 0x2 ref 0 00000100:00100000:0.0:1789760714.878546:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760714.879537:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495745664 00000100:00100000:2.0:1789760714.879549:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495745664 00000100:00100000:3.0:1789760714.879684:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699076087499 time 699145304292 state 0x0 ref 1 00000100:00100000:3.0:1789760714.879701:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699145564708 time 699145304292 state 0x1 ref 1 00000100:00100000:3.0:1789760714.879736:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699145564708 time 699200452302 state 0x0 ref 1 00000100:00100000:3.0:1789760714.879748:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495745664:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760714.879795:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21352448, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:3.0:1789760714.879803:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545657 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.879820:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760714.879825:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.879831:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545657 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.879845:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545657 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760714.921374:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21352448 -> 21356544 00010000:00010000:3.0:1789760714.921393:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545657 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.921407:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545657 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.921417:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545657 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.921424:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760714.921429:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545657 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.921439:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545657 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.921458:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545657 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.921466:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545657 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.921562:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495745664:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41809us (43018us total) trans 4294972530 rc 0/0 00000100:00100000:3.0:1789760714.921574:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5242 00000100:00100000:3.0:1789760714.921584:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699145564708 time 699200452302 state 0x2 ref 0 00000001:00080000:0.0:1789760714.921664:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972530 is committed 00000100:00100000:1.0:1789760714.926596:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760714.926943:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495745792 00000100:00100000:2.0:1789760714.926953:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495745792 00000100:00100000:2.0:1789760714.927007:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699145564708 time 699200452302 state 0x0 ref 1 00000100:00100000:2.0:1789760714.927022:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699200712718 time 699200452302 state 0x1 ref 1 00000100:00100000:2.0:1789760714.927056:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699200712718 time 699247772790 state 0x0 ref 1 00000100:00100000:2.0:1789760714.927068:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495745792:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760714.927119:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21356544, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:2.0:1789760714.927129:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454565e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.927149:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:2.0:1789760714.927153:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.927159:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454565e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.927171:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454565e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760714.972167:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972531 is committed 00002000:00010000:2.0:1789760714.972253:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21356544 -> 21360640 00010000:00010000:2.0:1789760714.972269:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454565e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.972300:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454565e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.972309:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454565e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.972315:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760714.972321:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454565e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.972331:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454565e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.972356:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454565e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.972365:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454565e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760714.972474:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495745792:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45403us (45883us total) trans 4294972531 rc 0/0 00000100:00100000:2.0:1789760714.972485:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5243 00000100:00100000:2.0:1789760714.972495:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699200712718 time 699247772790 state 0x2 ref 0 00000100:00100000:2.0:1789760714.979033:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760714.979172:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495745920 00000100:00100000:3.0:1789760714.979181:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495745920 00000100:00100000:3.0:1789760714.979247:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699200712718 time 699247772790 state 0x0 ref 1 00000100:00100000:3.0:1789760714.979262:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699248033206 time 699247772790 state 0x1 ref 1 00000100:00100000:3.0:1789760714.979326:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699248033206 time 699300042074 state 0x0 ref 1 00000100:00100000:3.0:1789760714.979338:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495745920:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760714.979384:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21360640, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:3.0:1789760714.979392:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545665 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.979406:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760714.979410:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.979416:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545665 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.979429:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545665 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760715.038495:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789760715.038519:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff92900c6a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520301824:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760715.038604:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760715.038996:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520301824 00000100:00100000:1.0:1789760715.039004:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520301824 00000100:00100000:1.0:1789760715.039074:0:6711:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 189 00000100:00100000:1.0:1789760715.039084:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520301824:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760715.039108:0:6711:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527761408 pending 1703936 free 3836739584 avail 3829399552 00000100:00100000:1.0:1789760715.039161:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520301824:12345-0@lo:13:osp-pre-0-0.0 Request processed in 79us (557us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760715.039168:0:6711:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 189 00000100:00100000:1.0:1789760715.039213:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff92900c6a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520301824:0@lo:13:osp-pre-0-0.0 00000100:00080000:2.0:1789760715.039432:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760715.043523:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972532 is committed 00002000:00010000:2.0:1789760715.043638:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21360640 -> 21364736 00010000:00010000:2.0:1789760715.043653:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545665 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.043666:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545665 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.043675:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545665 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.043684:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760715.043689:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545665 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.043700:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545665 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.043719:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545665 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.043726:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545665 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760715.043815:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495745920:12345-192.168.203.9@tcp:4:dd.0 Request processed in 64474us (64785us total) trans 4294972532 rc 0/0 00000100:00100000:2.0:1789760715.043826:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5244 00000100:00100000:2.0:1789760715.043835:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699248033206 time 699300042074 state 0x2 ref 0 00000100:00100000:0.0:1789760715.049111:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760715.049199:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495746048 00000100:00100000:3.0:1789760715.049209:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495746048 00000100:00080000:2.0:1789760715.049269:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760715.049327:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699248033206 time 699300042074 state 0x0 ref 1 00000100:00100000:3.0:1789760715.049347:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699300302490 time 699300042074 state 0x1 ref 1 00000100:00100000:3.0:1789760715.049373:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699300302490 time 699370089913 state 0x0 ref 1 00000100:00100000:3.0:1789760715.049387:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495746048:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760715.049435:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21364736, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:3.0:1789760715.049443:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454566c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.049459:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760715.049463:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760715.049469:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454566c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.049482:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454566c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760715.100632:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972533 is committed 00002000:00010000:3.0:1789760715.101183:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21364736 -> 21368832 00010000:00010000:3.0:1789760715.101200:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454566c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.101213:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454566c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.101439:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454566c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.101452:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760715.101457:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454566c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.101465:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454566c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.101485:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454566c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.101493:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454566c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760715.101629:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495746048:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52241us (52523us total) trans 4294972533 rc 0/0 00000100:00100000:3.0:1789760715.101641:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5245 00000100:00100000:3.0:1789760715.101651:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699300302490 time 699370089913 state 0x2 ref 0 00000100:00100000:1.0:1789760715.107113:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760715.107183:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495746176 00000100:00100000:2.0:1789760715.107192:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495746176 00000100:00100000:2.0:1789760715.107270:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699300302490 time 699370089913 state 0x0 ref 1 00000100:00100000:2.0:1789760715.107333:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699370350329 time 699370089913 state 0x1 ref 1 00000100:00100000:2.0:1789760715.107363:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699370350329 time 699428079075 state 0x0 ref 1 00000100:00100000:2.0:1789760715.107376:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495746176:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760715.107424:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21368832, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:2.0:1789760715.107433:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545673 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.107463:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:2.0:1789760715.107467:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.107474:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545673 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.107488:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545673 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760715.149725:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972534 is committed 00002000:00010000:3.0:1789760715.149841:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21368832 -> 21372928 00010000:00010000:3.0:1789760715.149863:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545673 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.149875:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545673 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.149888:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545673 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.149894:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760715.149899:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545673 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.149907:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545673 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.149924:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545673 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.149930:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545673 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760715.150053:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495746176:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42674us (42942us total) trans 4294972534 rc 0/0 00000100:00100000:3.0:1789760715.150064:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5246 00000100:00100000:3.0:1789760715.150074:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699370350329 time 699428079075 state 0x2 ref 0 00000100:00100000:2.0:1789760715.163187:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760715.163422:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495746304 00000100:00100000:0.0:1789760715.163431:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495746304 00000100:00100000:0.0:1789760715.163608:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699370350329 time 699428079075 state 0x0 ref 1 00000100:00100000:0.0:1789760715.163625:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699428339491 time 699428079075 state 0x1 ref 1 00000100:00100000:0.0:1789760715.163659:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699428339491 time 699484374760 state 0x0 ref 1 00000100:00100000:0.0:1789760715.163671:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495746304:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760715.163721:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21372928, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:0.0:1789760715.163730:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454567a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.163745:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:0.0:1789760715.163749:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760715.163756:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454567a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.163769:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454567a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760715.272030:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972535 is committed 00002000:00010000:1.0:1789760715.272514:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21372928 -> 21377024 00010000:00010000:1.0:1789760715.272530:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454567a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.272543:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454567a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.272552:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454567a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.272559:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760715.272564:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454567a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.272573:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454567a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.272593:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454567a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.272601:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454567a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760715.272695:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495746304:12345-192.168.203.9@tcp:4:dd.0 Request processed in 109021us (109510us total) trans 4294972535 rc 0/0 00000100:00100000:1.0:1789760715.272707:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5247 00000100:00100000:1.0:1789760715.272717:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699428339491 time 699484374760 state 0x2 ref 0 00000100:00100000:0.0:1789760715.275932:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760715.276019:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495746432 00000100:00100000:3.0:1789760715.276028:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495746432 00000100:00100000:3.0:1789760715.276080:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699428339491 time 699484374760 state 0x0 ref 1 00000100:00100000:3.0:1789760715.276094:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699484635176 time 699484374760 state 0x1 ref 1 00000100:00100000:3.0:1789760715.276163:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699484635176 time 699596879498 state 0x0 ref 1 00000100:00100000:3.0:1789760715.276176:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495746432:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760715.276247:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21377024, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:3.0:1789760715.276257:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545681 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.276271:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760715.276428:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760715.276435:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545681 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.276449:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545681 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760715.322928:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972536 is committed 00002000:00010000:3.0:1789760715.323381:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21377024 -> 21381120 00010000:00010000:3.0:1789760715.323397:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545681 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.323411:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545681 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.323421:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545681 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.323429:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760715.323434:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545681 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.323444:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545681 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.323465:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545681 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.323473:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545681 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760715.323601:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495746432:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47422us (47674us total) trans 4294972536 rc 0/0 00000100:00100000:3.0:1789760715.323612:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5248 00000100:00100000:3.0:1789760715.323624:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699484635176 time 699596879498 state 0x2 ref 0 00000100:00100000:1.0:1789760715.329213:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760715.329339:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495746560 00000100:00100000:3.0:1789760715.329349:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495746560 00000100:00100000:3.0:1789760715.329401:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699484635176 time 699596879498 state 0x0 ref 1 00000100:00100000:3.0:1789760715.329418:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699597139914 time 699596879498 state 0x1 ref 1 00000100:00100000:3.0:1789760715.329710:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699597139914 time 699650425383 state 0x0 ref 1 00000100:00100000:3.0:1789760715.329725:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495746560:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760715.329773:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21381120, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:3.0:1789760715.329783:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545688 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.329800:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760715.329805:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760715.329811:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545688 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.329824:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545688 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760715.355087:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21381120 -> 21385216 00010000:00010000:3.0:1789760715.355106:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545688 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.355119:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545688 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.355128:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545688 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.355135:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760715.355139:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545688 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.355148:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545688 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.355171:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545688 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.355180:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545688 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760715.355330:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495746560:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25602us (26121us total) trans 4294972537 rc 0/0 00000100:00100000:3.0:1789760715.355340:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5249 00000100:00100000:3.0:1789760715.355350:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699597139914 time 699650425383 state 0x2 ref 0 00000001:00080000:1.0:1789760715.355787:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972537 is committed 00000100:00100000:2.0:1789760715.360013:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760715.360109:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495746688 00000100:00100000:1.0:1789760715.360120:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495746688 00000100:00100000:1.0:1789760715.360177:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699597139914 time 699650425383 state 0x0 ref 1 00000100:00100000:1.0:1789760715.360192:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699650685799 time 699650425383 state 0x1 ref 1 00000100:00100000:1.0:1789760715.360232:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699650685799 time 699680949271 state 0x0 ref 1 00000100:00100000:1.0:1789760715.360254:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92901237ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495746688:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760715.360407:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21385216, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:1.0:1789760715.360416:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454568f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.360433:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:1.0:1789760715.360438:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760715.360444:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454568f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.360457:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454568f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760715.498695:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972538 is committed 00002000:00010000:3.0:1789760715.498870:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21385216 -> 21389312 00010000:00010000:3.0:1789760715.498887:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454568f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.498900:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454568f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.498909:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454568f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.498917:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760715.498922:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454568f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.498936:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454568f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.498957:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454568f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.498965:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454568f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760715.499061:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92901237ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495746688:12345-192.168.203.9@tcp:4:dd.0 Request processed in 138803us (139052us total) trans 4294972538 rc 0/0 00000100:00100000:3.0:1789760715.499072:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5250 00000100:00100000:3.0:1789760715.499082:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699650685799 time 699680949271 state 0x2 ref 0 00000100:00100000:2.0:1789760715.509439:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760715.509550:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495746816 00000100:00100000:3.0:1789760715.509566:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495746816 00000100:00100000:3.0:1789760715.509621:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699650685799 time 699680949271 state 0x0 ref 1 00000100:00100000:3.0:1789760715.509636:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699681209687 time 699680949271 state 0x1 ref 1 00000100:00100000:3.0:1789760715.509670:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699681209687 time 699830386790 state 0x0 ref 1 00000100:00100000:3.0:1789760715.509682:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04931f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495746816:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760715.509731:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21389312, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:3.0:1789760715.509740:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545696 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.509756:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760715.509761:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760715.509768:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545696 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.509782:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545696 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760715.599711:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972539 is committed 00002000:00010000:3.0:1789760715.599792:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21389312 -> 21393408 00010000:00010000:3.0:1789760715.599807:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545696 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.599820:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545696 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.599830:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545696 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.599838:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760715.599843:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545696 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.599852:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545696 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.599873:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545696 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.599882:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545696 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760715.599971:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495746816:12345-192.168.203.9@tcp:4:dd.0 Request processed in 90284us (90536us total) trans 4294972539 rc 0/0 00000100:00100000:3.0:1789760715.599981:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5251 00000100:00100000:3.0:1789760715.599991:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699681209687 time 699830386790 state 0x2 ref 0 00000100:00100000:1.0:1789760715.603204:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760715.603422:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495746944 00000100:00100000:3.0:1789760715.603430:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495746944 00000100:00100000:2.0:1789760715.603541:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699681209687 time 699830386790 state 0x0 ref 1 00000100:00100000:2.0:1789760715.603623:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699830647206 time 699830386790 state 0x1 ref 1 00000100:00100000:2.0:1789760715.603652:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699830647206 time 699924369632 state 0x0 ref 1 00000100:00100000:2.0:1789760715.603664:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495746944:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760715.603709:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21393408, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:2.0:1789760715.603718:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454569d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.603734:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:2.0:1789760715.603739:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.603746:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454569d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.603760:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454569d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.616765:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760715.671075:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972540 is committed 00002000:00010000:3.0:1789760715.671187:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21393408 -> 21397504 00010000:00010000:3.0:1789760715.671202:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454569d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.671308:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454569d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.671318:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454569d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.671326:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760715.671331:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454569d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.671341:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454569d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.671366:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454569d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.671374:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454569d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760715.671478:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495746944:12345-192.168.203.9@tcp:4:dd.0 Request processed in 67811us (68276us total) trans 4294972540 rc 0/0 00000100:00100000:3.0:1789760715.671488:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5252 00000100:00100000:3.0:1789760715.671499:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699830647206 time 699924369632 state 0x2 ref 0 00000100:00100000:2.0:1789760715.676915:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760715.677364:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495747072 00000100:00100000:3.0:1789760715.677374:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495747072 00000100:00100000:3.0:1789760715.677426:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699830647206 time 699924369632 state 0x0 ref 1 00000100:00100000:3.0:1789760715.677442:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699924630048 time 699924369632 state 0x1 ref 1 00000100:00100000:3.0:1789760715.677510:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699924630048 time 699998226794 state 0x0 ref 1 00000100:00100000:3.0:1789760715.677523:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04933480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495747072:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760715.677577:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21397504, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:3.0:1789760715.677586:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456a4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.677603:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760715.677608:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760715.677615:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.677628:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760715.725774:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21397504 -> 21401600 00002000:00010000:3.0:1789760715.725784:0:6714:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760714 -> 1789760715 00002000:00010000:3.0:1789760715.725786:0:6714:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760714 -> 1789760715 00010000:00010000:3.0:1789760715.725799:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.725812:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456a4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.725821:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.725828:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760715.725833:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.725982:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456a4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.726005:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456a4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.726014:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456a4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760715.726106:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04933480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495747072:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48580us (49194us total) trans 4294972541 rc 0/0 00000100:00100000:3.0:1789760715.726118:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5253 00000100:00100000:3.0:1789760715.726128:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699924630048 time 699998226794 state 0x2 ref 0 00000001:00080000:1.0:1789760715.736018:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972541 is committed 00000100:00100000:0.0:1789760715.736481:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760715.736626:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495747200 00000100:00100000:3.0:1789760715.736635:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495747200 00000100:00100000:3.0:1789760715.736685:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699924630048 time 699998226794 state 0x0 ref 1 00000100:00100000:3.0:1789760715.736700:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699998487210 time 699998226794 state 0x1 ref 1 00000100:00100000:3.0:1789760715.736770:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699998487210 time 700057485660 state 0x0 ref 1 00000100:00100000:3.0:1789760715.736783:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495747200:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760715.736829:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21401600, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa807 00010000:00010000:3.0:1789760715.736847:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456ab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.736864:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760715.736869:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760715.736875:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.736895:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760715.802066:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972542 is committed 00002000:00010000:3.0:1789760715.802249:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21401600 -> 21405696 00010000:00010000:3.0:1789760715.802265:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.802309:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456ab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.802320:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.802328:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760715.802333:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.802343:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.802362:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456ab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.802371:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456ab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760715.802473:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495747200:12345-192.168.203.9@tcp:4:dd.0 Request processed in 65687us (65993us total) trans 4294972542 rc 0/0 00000100:00100000:3.0:1789760715.802483:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5254 00000100:00100000:3.0:1789760715.802493:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699998487210 time 700057485660 state 0x2 ref 0 00000100:00100000:1.0:1789760715.807897:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760715.807968:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495747328 00000100:00100000:3.0:1789760715.807977:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495747328 00000100:00100000:3.0:1789760715.808035:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 699998487210 time 700057485660 state 0x0 ref 1 00000100:00100000:3.0:1789760715.808049:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700057746076 time 700057485660 state 0x1 ref 1 00000100:00100000:3.0:1789760715.808077:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700057746076 time 700128794279 state 0x0 ref 1 00000100:00100000:3.0:1789760715.808088:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495747328:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760715.808132:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21405696, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa807 00010000:00010000:3.0:1789760715.808141:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456b2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.808157:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760715.808161:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760715.808166:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.808179:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760715.916954:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972543 is committed 00002000:00010000:3.0:1789760715.917070:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21405696 -> 21409792 00010000:00010000:3.0:1789760715.917086:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.917100:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456b2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.917110:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.917118:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760715.917123:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.917132:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456b2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.917150:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456b2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.917158:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456b2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760715.917258:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495747328:12345-192.168.203.9@tcp:4:dd.0 Request processed in 109167us (109366us total) trans 4294972543 rc 0/0 00000100:00100000:3.0:1789760715.917271:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5255 00000100:00100000:3.0:1789760715.917414:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700057746076 time 700128794279 state 0x2 ref 0 00000100:00100000:3.0:1789760715.923891:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760715.924094:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495747456 00000100:00100000:0.0:1789760715.924105:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495747456 00000100:00100000:0.0:1789760715.924166:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700057746076 time 700128794279 state 0x0 ref 1 00000100:00100000:0.0:1789760715.924182:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700129054695 time 700128794279 state 0x1 ref 1 00000100:00100000:0.0:1789760715.924220:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700129054695 time 700244926328 state 0x0 ref 1 00000100:00100000:0.0:1789760715.924245:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495747456:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760715.924326:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21409792, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa807 00010000:00010000:0.0:1789760715.924336:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545456b9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.924353:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760715.924357:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760715.924363:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545456b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.924378:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545456b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760715.952649:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972544 is committed 00002000:00010000:2.0:1789760715.958567:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21409792 -> 21413888 00010000:00010000:2.0:1789760715.958587:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545456b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.958601:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545456b9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.958611:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545456b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.958618:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760715.958623:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545456b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.958632:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545456b9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.958652:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545456b9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.958665:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545456b9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760715.958778:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495747456:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34541us (34890us total) trans 4294972544 rc 0/0 00000100:00100000:2.0:1789760715.958789:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5256 00000100:00100000:2.0:1789760715.958798:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700129054695 time 700244926328 state 0x2 ref 0 00000100:00100000:3.0:1789760715.962438:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760715.962542:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495747584 00000100:00100000:0.0:1789760715.962555:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495747584 00000100:00100000:0.0:1789760715.962611:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700129054695 time 700244926328 state 0x0 ref 1 00000100:00100000:0.0:1789760715.962626:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700245186744 time 700244926328 state 0x1 ref 1 00000100:00100000:0.0:1789760715.962659:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700245186744 time 700283375909 state 0x0 ref 1 00000100:00100000:0.0:1789760715.962672:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495747584:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760715.962723:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21413888, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa807 00010000:00010000:0.0:1789760715.962732:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545456c0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.962749:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760715.962753:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760715.962759:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545456c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.962774:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545456c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760716.011756:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972545 is committed 00002000:00010000:3.0:1789760716.011938:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21413888 -> 21417984 00010000:00010000:3.0:1789760716.011963:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545456c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.011976:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545456c0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.011985:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545456c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.011992:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760716.011997:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545456c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.012006:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545456c0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.012027:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545456c0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.012035:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545456c0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760716.012141:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495747584:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49465us (49708us total) trans 4294972545 rc 0/0 00000100:00100000:3.0:1789760716.012152:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5257 00000100:00100000:3.0:1789760716.012162:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700245186744 time 700283375909 state 0x2 ref 0 00000100:00100000:1.0:1789760716.018380:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760716.018507:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495747712 00000100:00100000:3.0:1789760716.018517:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495747712 00000100:00100000:3.0:1789760716.018574:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700245186744 time 700283375909 state 0x0 ref 1 00000100:00100000:3.0:1789760716.018590:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700283636325 time 700283375909 state 0x1 ref 1 00000100:00100000:3.0:1789760716.018620:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700283636325 time 700339336054 state 0x0 ref 1 00000100:00100000:3.0:1789760716.018633:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495747712:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760716.018680:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21417984, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa807 00010000:00010000:3.0:1789760716.018689:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456c7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.018705:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760716.018710:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760716.018717:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.018732:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760716.078881:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21417984 -> 21422080 00010000:00010000:3.0:1789760716.078902:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.078915:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456c7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.078923:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.078930:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760716.078935:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.078943:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456c7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.078971:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456c7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.078979:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545456c7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760716.079062:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495747712:12345-192.168.203.9@tcp:4:dd.0 Request processed in 60427us (60687us total) trans 4294972546 rc 0/0 00000100:00100000:3.0:1789760716.079075:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5258 00000100:00100000:3.0:1789760716.079085:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700283636325 time 700339336054 state 0x2 ref 0 00000001:00080000:3.0:1789760716.079127:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972546 is committed 00000100:00100000:2.0:1789760716.083137:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760716.083193:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495747840 00000100:00100000:3.0:1789760716.083204:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495747840 00000100:00100000:3.0:1789760716.083271:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700283636325 time 700339336054 state 0x0 ref 1 00000100:00100000:3.0:1789760716.083336:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700339596470 time 700339336054 state 0x1 ref 1 00000100:00100000:3.0:1789760716.083369:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700339596470 time 700404085887 state 0x0 ref 1 00000100:00100000:3.0:1789760716.083382:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04932680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495747840:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760716.083427:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21422080, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa807 00010000:00010000:3.0:1789760716.083437:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456ce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.083455:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760716.083460:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760716.083467:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.083482:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760716.162331:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972547 is committed 00002000:00010000:3.0:1789760716.169340:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21422080 -> 21426176 00010000:00010000:3.0:1789760716.169364:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.169376:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456ce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.169384:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.169390:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760716.169395:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.169403:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456ce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.169423:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456ce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.169430:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456ce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760716.169534:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04932680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495747840:12345-192.168.203.9@tcp:4:dd.0 Request processed in 86151us (86402us total) trans 4294972547 rc 0/0 00000100:00100000:3.0:1789760716.169545:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5259 00000100:00100000:3.0:1789760716.169555:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700339596470 time 700404085887 state 0x2 ref 0 00000100:00100000:3.0:1789760716.173563:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760716.173669:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495747968 00000100:00100000:0.0:1789760716.173680:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495747968 00000100:00100000:0.0:1789760716.173743:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700339596470 time 700404085887 state 0x0 ref 1 00000100:00100000:0.0:1789760716.173761:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700404346303 time 700404085887 state 0x1 ref 1 00000100:00100000:0.0:1789760716.173795:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700404346303 time 700494511834 state 0x0 ref 1 00000100:00100000:0.0:1789760716.173807:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495747968:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760716.173860:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21426176, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa807 00010000:00010000:0.0:1789760716.173869:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545456d5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.173886:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760716.173890:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760716.173897:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545456d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.173910:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545456d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760716.218139:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972548 is committed 00002000:00010000:0.0:1789760716.218578:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21426176 -> 21430272 00010000:00010000:0.0:1789760716.218593:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545456d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.218607:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545456d5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.218616:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545456d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.218623:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760716.218628:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545456d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.218637:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545456d5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.218656:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545456d5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.218663:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545456d5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760716.218765:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495747968:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44955us (45208us total) trans 4294972548 rc 0/0 00000100:00100000:0.0:1789760716.218775:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5260 00000100:00100000:0.0:1789760716.218784:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700404346303 time 700494511834 state 0x2 ref 0 00000100:00100000:1.0:1789760716.223576:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760716.223667:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495748096 00000100:00100000:3.0:1789760716.223678:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495748096 00000100:00100000:3.0:1789760716.223738:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700404346303 time 700494511834 state 0x0 ref 1 00000100:00100000:3.0:1789760716.223755:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700494772250 time 700494511834 state 0x1 ref 1 00000100:00100000:3.0:1789760716.223780:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700494772250 time 700544496709 state 0x0 ref 1 00000100:00100000:3.0:1789760716.223792:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495748096:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760716.223841:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21430272, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa807 00010000:00010000:3.0:1789760716.223849:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456dc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.223865:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760716.223869:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760716.223875:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.223888:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760716.280355:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972549 is committed 00002000:00010000:3.0:1789760716.280404:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21430272 -> 21434368 00010000:00010000:3.0:1789760716.280424:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.280443:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456dc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.280452:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.280459:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760716.280464:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.280473:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456dc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.280493:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456dc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.280501:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456dc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760716.280605:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495748096:12345-192.168.203.9@tcp:4:dd.0 Request processed in 56810us (57033us total) trans 4294972549 rc 0/0 00000100:00100000:3.0:1789760716.280614:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5261 00000100:00100000:3.0:1789760716.280624:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700494772250 time 700544496709 state 0x2 ref 0 00000100:00100000:2.0:1789760716.285805:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760716.286396:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495748224 00000100:00100000:3.0:1789760716.286408:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495748224 00000100:00100000:3.0:1789760716.286468:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700494772250 time 700544496709 state 0x0 ref 1 00000100:00100000:3.0:1789760716.286483:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700544757125 time 700544496709 state 0x1 ref 1 00000100:00100000:3.0:1789760716.286518:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700544757125 time 700607234678 state 0x0 ref 1 00000100:00100000:3.0:1789760716.286532:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04930a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495748224:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760716.286595:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21434368, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa807 00010000:00010000:3.0:1789760716.286605:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456e3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.286622:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760716.286627:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760716.286634:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.286648:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760716.351861:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972550 is committed 00002000:00010000:3.0:1789760716.351999:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21434368 -> 21438464 00010000:00010000:3.0:1789760716.352015:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.352027:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456e3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.352042:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.352049:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760716.352054:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.352063:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456e3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.352092:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456e3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.352100:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456e3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760716.352200:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04930a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495748224:12345-192.168.203.9@tcp:4:dd.0 Request processed in 65664us (66399us total) trans 4294972550 rc 0/0 00000100:00100000:3.0:1789760716.352346:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5262 00000100:00100000:3.0:1789760716.352356:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700544757125 time 700607234678 state 0x2 ref 0 00000100:00100000:0.0:1789760716.356997:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760716.357079:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495748352 00000100:00100000:3.0:1789760716.357089:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495748352 00000100:00100000:3.0:1789760716.357142:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700544757125 time 700607234678 state 0x0 ref 1 00000100:00100000:3.0:1789760716.357157:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700607495094 time 700607234678 state 0x1 ref 1 00000100:00100000:3.0:1789760716.357188:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700607495094 time 700677904631 state 0x0 ref 1 00000100:00100000:3.0:1789760716.357199:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495748352:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760716.357265:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21438464, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa807 00010000:00010000:3.0:1789760716.357275:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456ea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.357329:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760716.357334:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760716.357341:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.357356:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760716.419968:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972551 is committed 00002000:00010000:3.0:1789760716.420191:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21438464 -> 21442560 00010000:00010000:3.0:1789760716.420216:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.420238:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456ea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.420246:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.420253:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760716.420259:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.420268:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456ea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.420307:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456ea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.420316:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456ea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760716.420404:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495748352:12345-192.168.203.9@tcp:4:dd.0 Request processed in 63202us (63412us total) trans 4294972551 rc 0/0 00000100:00100000:3.0:1789760716.420414:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5263 00000100:00100000:3.0:1789760716.420424:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700607495094 time 700677904631 state 0x2 ref 0 00000100:00100000:0.0:1789760716.424908:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760716.424989:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495748480 00000100:00100000:3.0:1789760716.424998:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495748480 00000100:00100000:3.0:1789760716.425053:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700607495094 time 700677904631 state 0x0 ref 1 00000100:00100000:3.0:1789760716.425067:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700678165047 time 700677904631 state 0x1 ref 1 00000100:00100000:3.0:1789760716.425098:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700678165047 time 700745814982 state 0x0 ref 1 00000100:00100000:3.0:1789760716.425110:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495748480:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760716.425157:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21442560, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa807 00010000:00010000:3.0:1789760716.425166:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456f1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.425183:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760716.425188:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760716.425195:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.425222:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760716.466082:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21442560 -> 21446656 00010000:00010000:3.0:1789760716.466102:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.466115:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456f1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.466124:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.466131:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760716.466136:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.466145:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456f1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.466171:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456f1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.466179:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456f1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760716.466333:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495748480:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41219us (41429us total) trans 4294972552 rc 0/0 00000100:00100000:3.0:1789760716.466344:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5264 00000100:00100000:3.0:1789760716.466354:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700678165047 time 700745814982 state 0x2 ref 0 00000001:00080000:0.0:1789760716.466592:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972552 is committed 00000100:00100000:2.0:1789760716.470877:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760716.470954:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495748608 00000100:00100000:3.0:1789760716.470963:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495748608 00000100:00100000:3.0:1789760716.471015:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700678165047 time 700745814982 state 0x0 ref 1 00000100:00100000:3.0:1789760716.471029:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700746075398 time 700745814982 state 0x1 ref 1 00000100:00100000:3.0:1789760716.471057:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700746075398 time 700791773475 state 0x0 ref 1 00000100:00100000:3.0:1789760716.471069:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04931180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495748608:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760716.471115:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21446656, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa807 00010000:00010000:3.0:1789760716.471124:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456f8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.471140:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760716.471144:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760716.471151:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.471165:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760716.510408:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972553 is committed 00002000:00010000:3.0:1789760716.511626:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21446656 -> 21450752 00010000:00010000:3.0:1789760716.511643:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.511656:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456f8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.511666:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.511673:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760716.511678:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.511687:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456f8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.511708:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456f8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.511716:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545456f8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760716.511806:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495748608:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40735us (40934us total) trans 4294972553 rc 0/0 00000100:00100000:3.0:1789760716.511816:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5265 00000100:00100000:3.0:1789760716.511826:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700746075398 time 700791773475 state 0x2 ref 0 00000100:00100000:2.0:1789760716.515900:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760716.515985:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495748736 00000100:00100000:3.0:1789760716.515995:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495748736 00000100:00100000:3.0:1789760716.516045:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700746075398 time 700791773475 state 0x0 ref 1 00000100:00100000:3.0:1789760716.516061:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700792033891 time 700791773475 state 0x1 ref 1 00000100:00100000:3.0:1789760716.516095:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700792033891 time 700836810857 state 0x0 ref 1 00000100:00100000:3.0:1789760716.516107:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04932300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495748736:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760716.516150:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21450752, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa807 00010000:00010000:3.0:1789760716.516158:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456ff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.516174:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760716.516178:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760716.516184:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.516198:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760716.574561:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972554 is committed 00002000:00010000:2.0:1789760716.578156:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21450752 -> 21454848 00010000:00010000:2.0:1789760716.578181:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.578197:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456ff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.578218:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.578234:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760716.578240:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.578250:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456ff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.578463:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456ff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.578472:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545456ff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760716.578585:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04932300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495748736:12345-192.168.203.9@tcp:4:dd.0 Request processed in 62474us (62688us total) trans 4294972554 rc 0/0 00000100:00100000:2.0:1789760716.578597:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5266 00000100:00100000:2.0:1789760716.578609:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700792033891 time 700836810857 state 0x2 ref 0 00000100:00100000:0.0:1789760716.584846:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760716.584936:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495748864 00000100:00100000:3.0:1789760716.584946:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495748864 00000100:00100000:3.0:1789760716.585000:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700792033891 time 700836810857 state 0x0 ref 1 00000100:00100000:3.0:1789760716.585014:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700837071273 time 700836810857 state 0x1 ref 1 00000100:00100000:3.0:1789760716.585042:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700837071273 time 700905759400 state 0x0 ref 1 00000100:00100000:3.0:1789760716.585053:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495748864:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760716.585103:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21454848, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa807 00010000:00010000:3.0:1789760716.585113:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545706 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.585130:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760716.585135:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760716.585142:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545706 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.585156:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545706 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.653563:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760716.670166:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972555 is committed 00002000:00010000:2.0:1789760716.670179:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21454848 -> 21458944 00010000:00010000:2.0:1789760716.670196:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545706 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.670219:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545706 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.670237:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545706 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.670246:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760716.670251:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545706 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.670262:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545706 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.670318:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545706 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.670328:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545706 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760716.670422:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495748864:12345-192.168.203.9@tcp:4:dd.0 Request processed in 85366us (85581us total) trans 4294972555 rc 0/0 00000100:00100000:2.0:1789760716.670433:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5267 00000100:00100000:2.0:1789760716.670445:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700837071273 time 700905759400 state 0x2 ref 0 00000100:00100000:0.0:1789760716.674777:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760716.674870:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495748992 00000100:00100000:3.0:1789760716.674879:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495748992 00000100:00100000:3.0:1789760716.674934:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700837071273 time 700905759400 state 0x0 ref 1 00000100:00100000:3.0:1789760716.674949:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700906019816 time 700905759400 state 0x1 ref 1 00000100:00100000:3.0:1789760716.674983:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700906019816 time 700995699546 state 0x0 ref 1 00000100:00100000:3.0:1789760716.674994:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495748992:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760716.675046:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21458944, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa807 00010000:00010000:3.0:1789760716.675055:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454570d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.675071:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760716.675076:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760716.675084:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454570d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.675097:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454570d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760716.729803:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21458944 -> 21463040 00002000:00010000:2.0:1789760716.729814:0:6714:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760715 -> 1789760716 00002000:00010000:2.0:1789760716.729817:0:6714:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760715 -> 1789760716 00010000:00010000:2.0:1789760716.729833:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454570d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.729848:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454570d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.729857:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454570d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.729865:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760716.729870:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454570d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.729879:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454570d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.729902:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454570d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.729910:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454570d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760716.730003:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495748992:12345-192.168.203.9@tcp:4:dd.0 Request processed in 55005us (55232us total) trans 4294972556 rc 0/0 00000100:00100000:2.0:1789760716.730014:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5268 00000100:00100000:2.0:1789760716.730024:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700906019816 time 700995699546 state 0x2 ref 0 00000001:00080000:1.0:1789760716.730132:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972556 is committed 00000100:00100000:0.0:1789760716.736653:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760716.736754:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495749120 00000100:00100000:2.0:1789760716.736764:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495749120 00000100:00100000:2.0:1789760716.736816:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700906019816 time 700995699546 state 0x0 ref 1 00000100:00100000:2.0:1789760716.736830:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700995959962 time 700995699546 state 0x1 ref 1 00000100:00100000:2.0:1789760716.736854:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700995959962 time 701057571319 state 0x0 ref 1 00000100:00100000:2.0:1789760716.736868:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903639c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495749120:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760716.736916:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21463040, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa807 00010000:00010000:2.0:1789760716.736925:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545714 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.736942:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:2.0:1789760716.736947:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760716.736953:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545714 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.736966:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545714 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760716.779653:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21463040 -> 21467136 00010000:00010000:2.0:1789760716.779673:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545714 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.779687:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545714 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.779697:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545714 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.779705:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760716.779710:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545714 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.779720:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545714 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.779740:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545714 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.779749:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545714 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760716.779843:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903639c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495749120:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42971us (43193us total) trans 4294972557 rc 0/0 00000100:00100000:2.0:1789760716.779853:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5269 00000100:00100000:2.0:1789760716.779862:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700995959962 time 701057571319 state 0x2 ref 0 00000001:00080000:1.0:1789760716.780894:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972557 is committed 00000100:00100000:2.0:1789760716.783838:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760716.783920:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495749248 00000100:00100000:3.0:1789760716.783929:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495749248 00000100:00100000:3.0:1789760716.783982:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 700995959962 time 701057571319 state 0x0 ref 1 00000100:00100000:3.0:1789760716.783996:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701057831735 time 701057571319 state 0x1 ref 1 00000100:00100000:3.0:1789760716.784024:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701057831735 time 701104741553 state 0x0 ref 1 00000100:00100000:3.0:1789760716.784035:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04931f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495749248:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760716.784081:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21467136, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa807 00010000:00010000:3.0:1789760716.784090:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454571b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.784105:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:3.0:1789760716.784109:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760716.784115:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454571b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.784128:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454571b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760716.881650:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972558 is committed 00002000:00010000:3.0:1789760716.882091:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21467136 -> 21471232 00010000:00010000:3.0:1789760716.882106:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454571b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.882118:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454571b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.882127:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454571b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.882134:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760716.882140:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454571b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.882150:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454571b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.882173:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454571b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.882180:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454571b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760716.882320:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495749248:12345-192.168.203.9@tcp:4:dd.0 Request processed in 98243us (98450us total) trans 4294972558 rc 0/0 00000100:00100000:3.0:1789760716.882331:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5270 00000100:00100000:3.0:1789760716.882341:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701057831735 time 701104741553 state 0x2 ref 0 00000100:00100000:3.0:1789760716.885468:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760716.887113:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495749376 00000100:00100000:2.0:1789760716.887126:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495749376 00000100:00100000:0.0:1789760716.892929:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701057831735 time 701104741553 state 0x0 ref 1 00000100:00100000:0.0:1789760716.892950:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701105001969 time 701104741553 state 0x1 ref 1 00000100:00100000:0.0:1789760716.892992:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701105001969 time 701213708569 state 0x0 ref 1 00000100:00100000:0.0:1789760716.893008:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495749376:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760716.893058:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21471232, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa807 00010000:00010000:0.0:1789760716.893068:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545722 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.893090:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760716.893094:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760716.893101:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545722 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.893115:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545722 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760716.960586:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972559 is committed 00002000:00010000:3.0:1789760716.960740:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21471232 -> 21475328 00010000:00010000:3.0:1789760716.960755:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545722 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.960768:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545722 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.960855:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545722 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.960863:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760716.960869:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545722 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.960882:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545722 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.960904:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545722 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.960912:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545722 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760716.961019:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495749376:12345-192.168.203.9@tcp:4:dd.0 Request processed in 68011us (75555us total) trans 4294972559 rc 0/0 00000100:00100000:3.0:1789760716.961030:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5271 00000100:00100000:3.0:1789760716.961040:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701105001969 time 701213708569 state 0x2 ref 0 00000100:00100000:0.0:1789760716.981364:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760716.981715:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495749504 00000100:00100000:2.0:1789760716.981726:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495749504 00000100:00100000:2.0:1789760716.981793:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701105001969 time 701213708569 state 0x0 ref 1 00000100:00100000:2.0:1789760716.981810:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701213968985 time 701213708569 state 0x1 ref 1 00000100:00100000:2.0:1789760716.981844:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701213968985 time 701302560243 state 0x0 ref 1 00000100:00100000:2.0:1789760716.981857:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495749504:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760716.981907:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21475328, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa807 00010000:00010000:2.0:1789760716.981916:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545729 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.981932:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:2.0:1789760716.981936:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760716.981948:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545729 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.981962:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545729 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760717.070907:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972560 is committed 00002000:00010000:3.0:1789760717.071075:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21475328 -> 21479424 00010000:00010000:3.0:1789760717.071091:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545729 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.071104:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545729 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.071113:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545729 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.071121:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760717.071127:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545729 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.071136:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545729 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.071155:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545729 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.071162:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545729 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760717.071410:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495749504:12345-192.168.203.9@tcp:4:dd.0 Request processed in 89550us (90050us total) trans 4294972560 rc 0/0 00000100:00100000:3.0:1789760717.071421:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5272 00000100:00100000:3.0:1789760717.071432:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701213968985 time 701302560243 state 0x2 ref 0 00000100:00100000:2.0:1789760717.076507:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760717.076627:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495749632 00000100:00100000:3.0:1789760717.076637:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495749632 00000100:00100000:3.0:1789760717.076708:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701213968985 time 701302560243 state 0x0 ref 1 00000100:00100000:3.0:1789760717.076723:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701302820659 time 701302560243 state 0x1 ref 1 00000100:00100000:3.0:1789760717.076757:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701302820659 time 701397473178 state 0x0 ref 1 00000100:00100000:3.0:1789760717.076770:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04933480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495749632:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760717.076815:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21479424, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa807 00010000:00010000:3.0:1789760717.076824:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545730 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.076840:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:3.0:1789760717.076844:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760717.076850:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545730 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.076863:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545730 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760717.077850:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760717.087864:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff929036f65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:59c4e264-9b75-482b-8813-3d1dc124f384:3317:1876699520301952:0@lo:400:kworker.0 00000100:00100000:1.0:1789760717.087932:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760717.087976:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff929036f65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3317:1876699520302464:0@lo:400:kworker.0 00000100:00100000:1.0:1789760717.088003:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760717.088030:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1789760717.088036:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff929036f64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3317:1876699520302080:0@lo:400:kworker.0 00000100:00100000:1.0:1789760717.088059:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760717.088077:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff929036f66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3317:1876699520302592:0@lo:400:kworker.0 00000100:00100000:1.0:1789760717.088099:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760717.088125:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff929036f65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520302208:0@lo:400:kworker.0 00000100:00100000:1.0:1789760717.088147:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760717.088165:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760717.088170:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff929036f66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520302336:0@lo:400:kworker.0 00000100:00100000:1.0:1789760717.088189:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760717.088242:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520301952 00000100:00100000:1.0:1789760717.088249:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520301952 00000100:00100000:1.0:1789760717.088314:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 391 00000100:00100000:1.0:1789760717.088324:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929012314850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:59c4e264-9b75-482b-8813-3d1dc124f384+19:3317:x1876699520301952:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760717.088376:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929012314850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:59c4e264-9b75-482b-8813-3d1dc124f384+19:3317:x1876699520301952:12345-0@lo:400:kworker.0 Request processed in 53us (443us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760717.088383:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 391 00000100:00100000:1.0:1789760717.088414:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520302464 00000100:00100000:1.0:1789760717.088419:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520302464 00000100:00100000:1.0:1789760717.088452:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520302080 00000100:00100000:1.0:1789760717.088455:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520302080 00000100:00100000:1.0:1789760717.088471:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520302208 00000100:00100000:1.0:1789760717.088473:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520302208 00000100:00100000:1.0:1789760717.088487:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 457 00000100:00100000:1.0:1789760717.088492:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1876699520302464:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760717.088525:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1876699520302464:12345-0@lo:400:kworker.0 Request processed in 32us (521us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760717.088531:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 457 00000100:00100000:1.0:1789760717.088542:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 458 00000100:00100000:1.0:1789760717.088548:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1876699520302080:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760717.088575:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1876699520302080:12345-0@lo:400:kworker.0 Request processed in 27us (515us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760717.088581:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 458 00000100:00100000:1.0:1789760717.088593:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 459 00000100:00100000:1.0:1789760717.088598:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520302208:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760717.088628:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520302208:12345-0@lo:400:kworker.0 Request processed in 30us (480us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760717.088634:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 459 00000100:00100000:1.0:1789760717.088653:0:10490:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520302592 00000100:00100000:1.0:1789760717.088657:0:10490:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520302592 00000100:00100000:1.0:1789760717.088682:0:10490:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520302336 00000100:00100000:1.0:1789760717.088685:0:10490:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520302336 00000100:00100000:1.0:1789760717.088704:0:10490:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 214 00000100:00100000:1.0:1789760717.088711:0:10490:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3317:x1876699520302592:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760717.088740:0:10490:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3317:x1876699520302592:12345-0@lo:400:kworker.0 Request processed in 30us (640us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760717.088746:0:10490:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 214 00000100:00100000:1.0:1789760717.088756:0:10490:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 215 00000100:00100000:1.0:1789760717.088762:0:10490:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520302336:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760717.088786:0:10490:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520302336:12345-0@lo:400:kworker.0 Request processed in 25us (596us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760717.088791:0:10490:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 215 00000100:00080000:1.0:1789760717.088814:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789760717.088818:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789760717.088820:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760717.088858:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff929036f66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520302336:0@lo:400:kworker.0 00000100:00100000:1.0:1789760717.088868:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff929036f65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520302208:0@lo:400:kworker.0 00000100:00100000:1.0:1789760717.088908:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff929036f64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3317:1876699520302080:0@lo:400:kworker.0 00000100:00100000:1.0:1789760717.088917:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff929036f66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3317:1876699520302592:0@lo:400:kworker.0 00000100:00100000:1.0:1789760717.088925:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff929036f65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:59c4e264-9b75-482b-8813-3d1dc124f384:3317:1876699520301952:0@lo:400:kworker.0 00000100:00100000:1.0:1789760717.088933:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff929036f65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3317:1876699520302464:0@lo:400:kworker.0 00000001:00080000:2.0:1789760717.113832:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972561 is committed 00002000:00010000:3.0:1789760717.113916:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21479424 -> 21483520 00010000:00010000:3.0:1789760717.113931:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545730 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.113945:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545730 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.113955:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545730 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.113963:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760717.113968:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545730 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.113978:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545730 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.114003:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545730 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.114011:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545730 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760717.114144:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04933480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495749632:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37370us (37640us total) trans 4294972561 rc 0/0 00000100:00100000:3.0:1789760717.114155:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5273 00000100:00100000:3.0:1789760717.114165:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701302820659 time 701397473178 state 0x2 ref 0 00000100:00100000:2.0:1789760717.118783:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760717.118874:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495749760 00000100:00100000:3.0:1789760717.118885:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495749760 00000100:00100000:3.0:1789760717.118939:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701302820659 time 701397473178 state 0x0 ref 1 00000100:00100000:3.0:1789760717.118956:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701397733594 time 701397473178 state 0x1 ref 1 00000100:00100000:3.0:1789760717.119025:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701397733594 time 701439741573 state 0x0 ref 1 00000100:00100000:3.0:1789760717.119038:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04932680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495749760:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760717.119087:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21483520, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa807 00010000:00010000:3.0:1789760717.119096:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545737 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.119111:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:3.0:1789760717.119116:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760717.119121:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545737 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.119135:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545737 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760717.167720:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972562 is committed 00002000:00010000:3.0:1789760717.167844:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21483520 -> 21487616 00010000:00010000:3.0:1789760717.167858:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545737 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.167871:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545737 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.167880:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545737 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.167887:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760717.167893:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545737 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.167903:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545737 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.167923:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545737 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.167931:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545737 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760717.168020:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04932680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495749760:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48978us (49244us total) trans 4294972562 rc 0/0 00000100:00100000:3.0:1789760717.168030:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5274 00000100:00100000:3.0:1789760717.168040:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701397733594 time 701439741573 state 0x2 ref 0 00000100:00100000:2.0:1789760717.172727:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760717.172817:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495749888 00000100:00100000:3.0:1789760717.172828:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495749888 00000100:00100000:3.0:1789760717.172882:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701397733594 time 701439741573 state 0x0 ref 1 00000100:00100000:3.0:1789760717.172898:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701440001989 time 701439741573 state 0x1 ref 1 00000100:00100000:3.0:1789760717.172932:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701440001989 time 701493648775 state 0x0 ref 1 00000100:00100000:3.0:1789760717.172945:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04930a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495749888:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760717.172989:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21487616, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa807 00010000:00010000:3.0:1789760717.172998:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454573e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.173014:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:3.0:1789760717.173018:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760717.173025:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454573e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.173039:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454573e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760717.240016:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972563 is committed 00002000:00010000:3.0:1789760717.240563:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21487616 -> 21491712 00010000:00010000:3.0:1789760717.240580:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454573e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.240594:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454573e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.240603:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454573e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.240610:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760717.240615:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454573e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.240624:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454573e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.240644:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454573e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.240652:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454573e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760717.240745:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04930a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495749888:12345-192.168.203.9@tcp:4:dd.0 Request processed in 67798us (68024us total) trans 4294972563 rc 0/0 00000100:00100000:3.0:1789760717.240758:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5275 00000100:00100000:3.0:1789760717.240768:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701440001989 time 701493648775 state 0x2 ref 0 00000100:00100000:2.0:1789760717.247582:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760717.247784:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495750016 00000100:00100000:3.0:1789760717.247794:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495750016 00000100:00100000:3.0:1789760717.247851:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701440001989 time 701493648775 state 0x0 ref 1 00000100:00100000:3.0:1789760717.247877:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701493909191 time 701493648775 state 0x1 ref 1 00000100:00100000:3.0:1789760717.247908:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701493909191 time 701568624713 state 0x0 ref 1 00000100:00100000:3.0:1789760717.247920:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04931180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495750016:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760717.247967:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21491712, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa807 00010000:00010000:3.0:1789760717.247975:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545745 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.247991:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:3.0:1789760717.247995:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760717.248001:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545745 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.248014:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545745 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760717.280821:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972564 is committed 00002000:00010000:3.0:1789760717.280824:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21491712 -> 21495808 00010000:00010000:3.0:1789760717.280845:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545745 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.280859:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545745 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.280877:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545745 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.280885:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760717.280889:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545745 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.280899:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545745 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.280922:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545745 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.280930:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545745 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760717.281030:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495750016:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33105us (33449us total) trans 4294972564 rc 0/0 00000100:00100000:3.0:1789760717.281042:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5276 00000100:00100000:3.0:1789760717.281054:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701493909191 time 701568624713 state 0x2 ref 0 00000100:00100000:2.0:1789760717.290905:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760717.290991:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495750144 00000100:00100000:3.0:1789760717.291001:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495750144 00000100:00100000:3.0:1789760717.291053:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701493909191 time 701568624713 state 0x0 ref 1 00000100:00100000:3.0:1789760717.291067:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701568885129 time 701568624713 state 0x1 ref 1 00000100:00100000:3.0:1789760717.291101:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701568885129 time 701611817356 state 0x0 ref 1 00000100:00100000:3.0:1789760717.291113:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04932300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495750144:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760717.291159:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21495808, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa807 00010000:00010000:3.0:1789760717.291168:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454574c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.291185:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:3.0:1789760717.291190:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760717.291196:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454574c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.291220:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454574c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760717.324764:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21495808 -> 21499904 00010000:00010000:3.0:1789760717.324784:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454574c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.324798:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454574c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.324812:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454574c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.324820:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760717.324825:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454574c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.324834:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454574c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.324853:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454574c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.325708:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454574c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760717.325814:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04932300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495750144:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34699us (34914us total) trans 4294972565 rc 0/0 00000100:00100000:3.0:1789760717.325825:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5277 00000100:00100000:3.0:1789760717.325834:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701568885129 time 701611817356 state 0x2 ref 0 00000100:00100000:2.0:1789760717.329933:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760717.330126:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495750272 00000100:00100000:3.0:1789760717.330137:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495750272 00000100:00100000:3.0:1789760717.330190:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701568885129 time 701611817356 state 0x0 ref 1 00000100:00100000:3.0:1789760717.330214:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701612077772 time 701611817356 state 0x1 ref 1 00000100:00100000:3.0:1789760717.330256:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701612077772 time 701650972988 state 0x0 ref 1 00000100:00100000:3.0:1789760717.330269:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04933b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495750272:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760717.330347:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21499904, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa807 00010000:00010000:3.0:1789760717.330356:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545753 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.330371:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:3.0:1789760717.330375:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760717.330382:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545753 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.330396:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545753 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760717.333477:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972565 is committed 00000001:00080000:0.0:1789760717.382040:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972566 is committed 00002000:00010000:3.0:1789760717.382237:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21499904 -> 21504000 00010000:00010000:3.0:1789760717.382252:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545753 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.382265:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545753 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.382274:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545753 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.382322:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760717.382328:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545753 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.382336:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545753 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.382355:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545753 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.382361:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545753 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760717.382441:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04933b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495750272:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52170us (52513us total) trans 4294972566 rc 0/0 00000100:00100000:3.0:1789760717.382450:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5278 00000100:00100000:3.0:1789760717.382461:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701612077772 time 701650972988 state 0x2 ref 0 00000100:00100000:2.0:1789760717.406088:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760717.406177:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495750400 00000100:00100000:3.0:1789760717.406187:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495750400 00000100:00100000:3.0:1789760717.406261:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701612077772 time 701650972988 state 0x0 ref 1 00000100:00100000:3.0:1789760717.406276:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701651233404 time 701650972988 state 0x1 ref 1 00000100:00100000:3.0:1789760717.406360:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701651233404 time 701727076208 state 0x0 ref 1 00000100:00100000:3.0:1789760717.406373:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04933100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495750400:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760717.406419:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21504000, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa807 00010000:00010000:3.0:1789760717.406430:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454575a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.406446:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:3.0:1789760717.406451:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760717.406457:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454575a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.406472:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454575a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760717.452326:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972567 is committed 00002000:00010000:0.0:1789760717.453994:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21504000 -> 21508096 00010000:00010000:0.0:1789760717.454013:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454575a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.454025:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454575a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.454033:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454575a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.454040:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760717.454045:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454575a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.454059:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454575a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.454078:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454575a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.454085:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454575a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.454385:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04933100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495750400:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48010us (48301us total) trans 4294972567 rc 0/0 00000100:00100000:0.0:1789760717.454396:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5279 00000100:00100000:0.0:1789760717.454406:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701651233404 time 701727076208 state 0x2 ref 0 00000100:00100000:0.0:1789760717.464567:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760717.464745:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495750528 00000100:00100000:3.0:1789760717.464755:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495750528 00000100:00100000:3.0:1789760717.464809:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701651233404 time 701727076208 state 0x0 ref 1 00000100:00100000:3.0:1789760717.464823:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701727336624 time 701727076208 state 0x1 ref 1 00000100:00100000:3.0:1789760717.464909:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701727336624 time 701785625076 state 0x0 ref 1 00000100:00100000:3.0:1789760717.464923:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495750528:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760717.464968:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21508096, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa807 00010000:00010000:3.0:1789760717.464977:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545761 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.464993:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:3.0:1789760717.464997:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760717.465004:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545761 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.465019:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545761 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760717.504667:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972568 is committed 00002000:00010000:1.0:1789760717.504774:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21508096 -> 21512192 00010000:00010000:1.0:1789760717.504789:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545761 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.504802:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545761 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.504811:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545761 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.504818:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760717.504823:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545761 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.504831:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545761 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.504849:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545761 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.504857:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545761 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760717.504950:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495750528:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40026us (40389us total) trans 4294972568 rc 0/0 00000100:00100000:1.0:1789760717.504961:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5280 00000100:00100000:1.0:1789760717.504973:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701727336624 time 701785625076 state 0x2 ref 0 00000100:00100000:2.0:1789760717.507962:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760717.508050:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495750656 00000100:00100000:3.0:1789760717.508060:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495750656 00000100:00100000:3.0:1789760717.508114:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701727336624 time 701785625076 state 0x0 ref 1 00000100:00100000:3.0:1789760717.508130:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701785885492 time 701785625076 state 0x1 ref 1 00000100:00100000:3.0:1789760717.508166:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701785885492 time 701828882625 state 0x0 ref 1 00000100:00100000:3.0:1789760717.508180:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04931c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495750656:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760717.508442:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21512192, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa807 00010000:00010000:3.0:1789760717.508452:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545768 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.508469:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:3.0:1789760717.508483:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760717.508490:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545768 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.508503:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545768 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760717.549653:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972569 is committed 00002000:00010000:2.0:1789760717.549714:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21512192 -> 21516288 00010000:00010000:2.0:1789760717.549727:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545768 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.549745:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545768 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.549754:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545768 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.549760:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760717.549766:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545768 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.549775:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545768 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.549795:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545768 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.549803:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545768 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760717.549895:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495750656:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41712us (41936us total) trans 4294972569 rc 0/0 00000100:00100000:2.0:1789760717.549904:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5281 00000100:00100000:2.0:1789760717.549914:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701785885492 time 701828882625 state 0x2 ref 0 00000100:00100000:3.0:1789760717.552433:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760717.552631:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495750784 00000100:00100000:0.0:1789760717.552641:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495750784 00000100:00100000:0.0:1789760717.552695:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701785885492 time 701828882625 state 0x0 ref 1 00000100:00100000:0.0:1789760717.552710:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701829143041 time 701828882625 state 0x1 ref 1 00000100:00100000:0.0:1789760717.552744:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701829143041 time 701873460170 state 0x0 ref 1 00000100:00100000:0.0:1789760717.552758:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495750784:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760717.552807:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21516288, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa807 00010000:00010000:0.0:1789760717.552816:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454576f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.552831:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760717.552836:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.552842:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454576f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.552855:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454576f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760717.599843:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789760717.599986:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520302720:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760717.600100:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760717.600170:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520302720 00000100:00100000:3.0:1789760717.600177:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520302720 00000100:00100000:3.0:1789760717.600230:0:6711:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 190 00000100:00100000:3.0:1789760717.600241:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520302720:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760717.600273:0:6711:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760717.600339:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520302720:12345-0@lo:13:osp-pre-1-0.0 Request processed in 99us (239us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760717.600346:0:6711:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 190 00000100:00100000:3.0:1789760717.600393:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520302720:0@lo:13:osp-pre-1-0.0 00000001:00080000:3.0:1789760717.603932:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972570 is committed 00002000:00010000:0.0:1789760717.604098:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21516288 -> 21520384 00010000:00010000:0.0:1789760717.604117:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454576f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.604129:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454576f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.604137:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454576f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.604142:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760717.604147:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454576f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.604155:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454576f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.604180:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454576f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.604186:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454576f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.604441:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495750784:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51676us (52007us total) trans 4294972570 rc 0/0 00000100:00100000:0.0:1789760717.604453:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5282 00000100:00100000:0.0:1789760717.604467:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701829143041 time 701873460170 state 0x2 ref 0 00000100:00100000:0.0:1789760717.615328:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760717.615399:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495750912 00000100:00100000:1.0:1789760717.615413:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495750912 00000100:00100000:1.0:1789760717.615497:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701829143041 time 701873460170 state 0x0 ref 1 00000100:00100000:1.0:1789760717.615522:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701873720586 time 701873460170 state 0x1 ref 1 00000100:00100000:1.0:1789760717.615559:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701873720586 time 701936275332 state 0x0 ref 1 00000100:00100000:1.0:1789760717.615575:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495750912:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760717.615639:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21520384, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa807 00010000:00010000:1.0:1789760717.615651:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545776 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.615673:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760717.615679:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760717.615687:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545776 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.615704:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545776 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760717.641437:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972571 is committed 00002000:00010000:1.0:1789760717.641480:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21520384 -> 21524480 00010000:00010000:1.0:1789760717.641501:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545776 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.641517:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545776 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.641528:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545776 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.641536:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760717.641542:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545776 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.641553:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545776 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.641580:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545776 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.641589:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545776 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760717.641713:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495750912:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26135us (26388us total) trans 4294972571 rc 0/0 00000100:00100000:1.0:1789760717.641730:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5283 00000100:00100000:1.0:1789760717.641743:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701873720586 time 701936275332 state 0x2 ref 0 00000100:00100000:3.0:1789760717.649186:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760717.649344:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495751040 00000100:00100000:2.0:1789760717.649355:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495751040 00000100:00100000:2.0:1789760717.649410:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701873720586 time 701936275332 state 0x0 ref 1 00000100:00100000:2.0:1789760717.649422:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701936535748 time 701936275332 state 0x1 ref 1 00000100:00100000:1.0:1789760717.649574:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701936535748 time 701970277974 state 0x0 ref 1 00000100:00100000:1.0:1789760717.649608:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495751040:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760717.649698:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21524480, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa807 00010000:00010000:1.0:1789760717.649709:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454577d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.649728:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760717.649733:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760717.649741:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454577d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.649757:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454577d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.663099:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760717.749760:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972572 is committed 00002000:00010000:0.0:1789760717.751931:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21524480 -> 21528576 00002000:00010000:0.0:1789760717.751939:0:13059:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760716 -> 1789760717 00002000:00010000:0.0:1789760717.751942:0:13059:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760716 -> 1789760717 00010000:00010000:0.0:1789760717.751955:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454577d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.751968:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454577d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.751977:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454577d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.751984:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760717.751989:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454577d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.751999:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454577d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.752021:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454577d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.752030:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454577d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.752187:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495751040:12345-192.168.203.9@tcp:4:dd.0 Request processed in 102588us (103059us total) trans 4294972572 rc 0/0 00000100:00100000:0.0:1789760717.752208:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5284 00000100:00100000:0.0:1789760717.752228:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701936535748 time 701970277974 state 0x2 ref 0 00000100:00100000:3.0:1789760717.757619:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760717.757913:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495751168 00000100:00100000:0.0:1789760717.757923:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495751168 00000100:00100000:0.0:1789760717.757983:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701936535748 time 701970277974 state 0x0 ref 1 00000100:00100000:0.0:1789760717.757998:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701970538390 time 701970277974 state 0x1 ref 1 00000100:00100000:0.0:1789760717.758032:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701970538390 time 702078748862 state 0x0 ref 1 00000100:00100000:0.0:1789760717.758045:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495751168:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760717.758090:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21528576, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa807 00010000:00010000:0.0:1789760717.758099:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545784 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.758116:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760717.758121:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.758127:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545784 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.758140:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545784 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760717.822179:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972573 is committed 00002000:00010000:0.0:1789760717.822351:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21528576 -> 21532672 00010000:00010000:0.0:1789760717.822367:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545784 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.822381:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545784 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.822391:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545784 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.822397:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760717.822403:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545784 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.822412:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545784 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.822437:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545784 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.822446:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545784 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.822654:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495751168:12345-192.168.203.9@tcp:4:dd.0 Request processed in 64606us (65039us total) trans 4294972573 rc 0/0 00000100:00100000:0.0:1789760717.822665:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5285 00000100:00100000:0.0:1789760717.822675:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701970538390 time 702078748862 state 0x2 ref 0 00000100:00100000:2.0:1789760717.830096:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760717.830187:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495751296 00000100:00100000:1.0:1789760717.830210:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495751296 00000100:00100000:1.0:1789760717.830274:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 701970538390 time 702078748862 state 0x0 ref 1 00000100:00100000:1.0:1789760717.830322:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702079009278 time 702078748862 state 0x1 ref 1 00000100:00100000:1.0:1789760717.830344:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702079009278 time 702151060972 state 0x0 ref 1 00000100:00100000:1.0:1789760717.830357:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f873480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495751296:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760717.830404:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21532672, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa807 00010000:00010000:1.0:1789760717.830413:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x493674835454578b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.830429:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760717.830433:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760717.830439:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x493674835454578b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.830453:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x493674835454578b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760717.898387:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972574 is committed 00002000:00010000:1.0:1789760717.898671:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21532672 -> 21536768 00010000:00010000:1.0:1789760717.898689:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x493674835454578b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.898702:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x493674835454578b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.898711:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x493674835454578b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.898718:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760717.898722:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x493674835454578b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.898732:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x493674835454578b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.898751:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x493674835454578b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.898758:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x493674835454578b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760717.898855:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f873480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495751296:12345-192.168.203.9@tcp:4:dd.0 Request processed in 68495us (68764us total) trans 4294972574 rc 0/0 00000100:00100000:1.0:1789760717.898865:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5286 00000100:00100000:1.0:1789760717.898875:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702079009278 time 702151060972 state 0x2 ref 0 00000100:00100000:2.0:1789760717.903703:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760717.904386:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495751424 00000100:00100000:0.0:1789760717.904397:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495751424 00000100:00100000:0.0:1789760717.904455:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702079009278 time 702151060972 state 0x0 ref 1 00000100:00100000:0.0:1789760717.904470:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702151321388 time 702151060972 state 0x1 ref 1 00000100:00100000:0.0:1789760717.904500:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702151321388 time 702225217040 state 0x0 ref 1 00000100:00100000:0.0:1789760717.904513:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f921c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495751424:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760717.904563:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21536768, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa807 00010000:00010000:0.0:1789760717.904571:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545792 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.904587:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760717.904592:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.904598:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545792 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.904611:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545792 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760717.986767:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972575 is committed 00002000:00010000:1.0:1789760717.986866:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21536768 -> 21540864 00010000:00010000:1.0:1789760717.986881:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545792 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.986893:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545792 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.986902:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545792 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.986909:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760717.986914:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545792 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.986922:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545792 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.986941:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545792 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.986948:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545792 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760717.987060:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f921c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495751424:12345-192.168.203.9@tcp:4:dd.0 Request processed in 82546us (83362us total) trans 4294972575 rc 0/0 00000100:00100000:1.0:1789760717.987071:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5287 00000100:00100000:1.0:1789760717.987081:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702151321388 time 702225217040 state 0x2 ref 0 00000100:00100000:3.0:1789760717.993497:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760717.993802:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495751552 00000100:00100000:0.0:1789760717.993812:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495751552 00000100:00100000:0.0:1789760717.993869:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702151321388 time 702225217040 state 0x0 ref 1 00000100:00100000:0.0:1789760717.993883:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702225477456 time 702225217040 state 0x1 ref 1 00000100:00100000:0.0:1789760717.993915:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702225477456 time 702314631547 state 0x0 ref 1 00000100:00100000:0.0:1789760717.993927:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01c8c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495751552:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760717.993971:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21540864, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa807 00010000:00010000:0.0:1789760717.993981:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545799 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.994006:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760717.994011:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.994018:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545799 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.994031:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545799 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760718.066170:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972576 is committed 00002000:00010000:0.0:1789760718.066731:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21540864 -> 21544960 00010000:00010000:0.0:1789760718.066748:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545799 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.066760:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545799 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.066769:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545799 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.066777:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94aa00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760718.066781:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545799 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.066790:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545799 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.066808:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545799 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.066816:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545799 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.066964:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01c8c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495751552:12345-192.168.203.9@tcp:4:dd.0 Request processed in 73034us (73470us total) trans 4294972576 rc 0/0 00000100:00100000:0.0:1789760718.066978:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5288 00000100:00100000:0.0:1789760718.066987:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702225477456 time 702314631547 state 0x2 ref 0 00000100:00100000:3.0:1789760718.094021:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760718.094763:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495751680 00000100:00100000:0.0:1789760718.094774:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495751680 00000100:00100000:0.0:1789760718.094826:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702225477456 time 702314631547 state 0x0 ref 1 00000100:00100000:0.0:1789760718.094840:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702314891963 time 702314631547 state 0x1 ref 1 00000100:00100000:0.0:1789760718.094870:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702314891963 time 702415587172 state 0x0 ref 1 00000100:00100000:0.0:1789760718.094881:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01c8d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495751680:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760718.094925:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21544960, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa807 00010000:00010000:0.0:1789760718.094934:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457a0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.094950:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760718.094954:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760718.094959:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.094972:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760718.129884:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21544960 -> 21549056 00010000:00010000:0.0:1789760718.129905:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.129919:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457a0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.129929:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.129937:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94aa00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760718.129942:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.129953:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457a0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.129972:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457a0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.129981:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457a0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.130080:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01c8d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495751680:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35195us (36062us total) trans 4294972577 rc 0/0 00000100:00100000:0.0:1789760718.130090:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5289 00000100:00100000:0.0:1789760718.130100:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702314891963 time 702415587172 state 0x2 ref 0 00000001:00080000:3.0:1789760718.130723:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972577 is committed 00000100:00100000:3.0:1789760718.135227:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760718.136598:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495751808 00000100:00100000:1.0:1789760718.136611:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495751808 00000100:00100000:1.0:1789760718.136674:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702314891963 time 702415587172 state 0x0 ref 1 00000100:00100000:1.0:1789760718.136690:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702415847588 time 702415587172 state 0x1 ref 1 00000100:00100000:1.0:1789760718.136717:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702415847588 time 702457433794 state 0x0 ref 1 00000100:00100000:1.0:1789760718.136730:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495751808:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760718.136782:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21549056, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa807 00010000:00010000:1.0:1789760718.136791:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545457a7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.136807:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760718.136812:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760718.136818:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545457a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.136835:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545457a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760718.174378:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972578 is committed 00002000:00010000:2.0:1789760718.174486:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21549056 -> 21553152 00010000:00010000:2.0:1789760718.174501:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545457a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.174514:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545457a7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.174523:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545457a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.174531:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760718.174535:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545457a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.174609:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545457a7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.174629:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545457a7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.174638:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545457a7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.174733:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495751808:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38001us (39521us total) trans 4294972578 rc 0/0 00000100:00100000:2.0:1789760718.174743:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5290 00000100:00100000:2.0:1789760718.174752:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702415847588 time 702457433794 state 0x2 ref 0 00000100:00100000:2.0:1789760718.180252:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760718.180949:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495751936 00000100:00100000:0.0:1789760718.180961:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495751936 00000100:00100000:1.0:1789760718.181168:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702415847588 time 702457433794 state 0x0 ref 1 00000100:00100000:1.0:1789760718.181184:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702457694210 time 702457433794 state 0x1 ref 1 00000100:00100000:1.0:1789760718.181409:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702457694210 time 702502123767 state 0x0 ref 1 00000100:00100000:1.0:1789760718.181424:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f923100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495751936:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760718.181471:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21553152, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa807 00010000:00010000:1.0:1789760718.181481:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545457ae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.181497:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760718.181502:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760718.181509:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545457ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.181524:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545457ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760718.222954:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972579 is committed 00002000:00010000:0.0:1789760718.223062:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21553152 -> 21557248 00010000:00010000:0.0:1789760718.223078:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545457ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.223092:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545457ae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.223102:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545457ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.223110:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760718.223115:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545457ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.223125:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545457ae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.223145:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545457ae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.223153:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545457ae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.223265:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f923100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495751936:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41838us (43077us total) trans 4294972579 rc 0/0 00000100:00100000:0.0:1789760718.223275:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5291 00000100:00100000:0.0:1789760718.223428:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702457694210 time 702502123767 state 0x2 ref 0 00000100:00100000:3.0:1789760718.226534:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760718.226766:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495752064 00000100:00100000:0.0:1789760718.226777:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495752064 00000100:00100000:1.0:1789760718.226877:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702457694210 time 702502123767 state 0x0 ref 1 00000100:00100000:1.0:1789760718.226892:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702502384183 time 702502123767 state 0x1 ref 1 00000100:00100000:1.0:1789760718.226923:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702502384183 time 702547639539 state 0x0 ref 1 00000100:00100000:1.0:1789760718.226936:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495752064:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760718.226985:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21557248, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa807 00010000:00010000:1.0:1789760718.226994:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545457b5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.227011:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760718.227016:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760718.227022:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545457b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.227036:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545457b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760718.269697:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21557248 -> 21561344 00010000:00010000:1.0:1789760718.269717:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545457b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.269730:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545457b5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.269739:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545457b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.269746:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760718.269751:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545457b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.269760:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545457b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.269780:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545457b5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.269789:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545457b5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760718.269883:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495752064:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42944us (43353us total) trans 4294972580 rc 0/0 00000100:00100000:1.0:1789760718.269894:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5292 00000100:00100000:1.0:1789760718.269904:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702502384183 time 702547639539 state 0x2 ref 0 00000001:00080000:0.0:1789760718.270105:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972580 is committed 00000100:00100000:3.0:1789760718.273429:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760718.273527:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495752192 00000100:00100000:1.0:1789760718.273545:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495752192 00000100:00100000:1.0:1789760718.273598:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702502384183 time 702547639539 state 0x0 ref 1 00000100:00100000:1.0:1789760718.273612:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702547899955 time 702547639539 state 0x1 ref 1 00000100:00100000:1.0:1789760718.273643:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702547899955 time 702594360471 state 0x0 ref 1 00000100:00100000:1.0:1789760718.273656:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495752192:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760718.273700:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21561344, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa807 00010000:00010000:1.0:1789760718.273708:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545457bc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.273722:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760718.273727:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760718.273732:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545457bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.273746:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545457bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760718.304676:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972581 is committed 00002000:00010000:1.0:1789760718.304919:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21561344 -> 21565440 00010000:00010000:1.0:1789760718.304935:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545457bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.304946:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545457bc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.304958:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545457bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.304964:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760718.304970:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545457bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.304978:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545457bc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.304998:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545457bc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.305006:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545457bc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760718.305099:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495752192:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31440us (31674us total) trans 4294972581 rc 0/0 00000100:00100000:1.0:1789760718.305110:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5293 00000100:00100000:1.0:1789760718.305120:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702547899955 time 702594360471 state 0x2 ref 0 00000100:00100000:2.0:1789760718.313732:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760718.313841:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495752320 00000100:00100000:0.0:1789760718.313853:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495752320 00000100:00100000:0.0:1789760718.313920:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702547899955 time 702594360471 state 0x0 ref 1 00000100:00100000:0.0:1789760718.313937:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702594620887 time 702594360471 state 0x1 ref 1 00000100:00100000:0.0:1789760718.314006:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702594620887 time 702634722322 state 0x0 ref 1 00000100:00100000:0.0:1789760718.314018:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f922300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495752320:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760718.314068:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21565440, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa807 00010000:00010000:0.0:1789760718.314080:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457c3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.314098:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760718.314103:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760718.314109:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.314123:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.323494:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760718.323561:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760718.323594:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760718.323797:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495752576 00000100:00100000:1.0:1789760718.323807:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495752576 00000100:00100000:1.0:1789760718.323872:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495752704 00000100:00100000:1.0:1789760718.323876:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495752704 00000100:00100000:1.0:1789760718.323902:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 460 00000100:00100000:1.0:1789760718.323912:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:17022965-1e4f-46fb-915c-61730096b8e2+8:2357:x1876699495752576:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:1.0:1789760718.323989:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:17022965-1e4f-46fb-915c-61730096b8e2+8:2357:x1876699495752576:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 78us (503us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760718.323997:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 460 00000100:00100000:1.0:1789760718.324014:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 461 00000100:00100000:1.0:1789760718.324021:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2357:x1876699495752704:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:1.0:1789760718.324049:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2357:x1876699495752704:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 29us (455us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760718.324055:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 461 00000100:00080000:3.0:1789760718.325107:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760718.325557:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495752448 00000100:00100000:0.0:1789760718.325567:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495752448 00000100:00100000:0.0:1789760718.325618:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 392 00000100:00100000:0.0:1789760718.325628:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929012314c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2357:x1876699495752448:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:0.0:1789760718.325698:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929012314c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2357:x1876699495752448:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 72us (2138us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760718.325706:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 392 00002000:00010000:0.0:1789760718.350482:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21565440 -> 21569536 00010000:00010000:0.0:1789760718.350624:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.350637:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457c3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.350646:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.350653:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94aa00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760718.350658:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.350666:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457c3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.350685:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457c3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.350692:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457c3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.350787:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f922300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495752320:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36767us (37062us total) trans 4294972582 rc 0/0 00000100:00100000:0.0:1789760718.350797:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5294 00000100:00100000:0.0:1789760718.350806:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702594620887 time 702634722322 state 0x2 ref 0 00000001:00080000:1.0:1789760718.354125:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972582 is committed 00000100:00100000:1.0:1789760718.355947:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760718.356141:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495752832 00000100:00100000:3.0:1789760718.356151:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495752832 00000100:00100000:3.0:1789760718.356224:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702594620887 time 702634722322 state 0x0 ref 1 00000100:00100000:3.0:1789760718.356239:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702634982738 time 702634722322 state 0x1 ref 1 00000100:00100000:3.0:1789760718.356269:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702634982738 time 702676985773 state 0x0 ref 1 00000100:00100000:3.0:1789760718.356512:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495752832:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760718.356570:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21569536, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa807 00010000:00010000:3.0:1789760718.356580:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545457ca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.356598:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760718.356603:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760718.356610:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545457ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.356625:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545457ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760718.396971:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21569536 -> 21573632 00010000:00010000:3.0:1789760718.396992:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545457ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.397005:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545457ca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.397014:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545457ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.397020:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760718.397025:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545457ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.397033:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545457ca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.397053:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545457ca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.397061:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545457ca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.397156:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495752832:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40644us (41213us total) trans 4294972583 rc 0/0 00000100:00100000:3.0:1789760718.397167:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5295 00000100:00100000:3.0:1789760718.397177:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702634982738 time 702676985773 state 0x2 ref 0 00000100:00100000:1.0:1789760718.403324:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760718.403363:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495752960 00000100:00100000:1.0:1789760718.403372:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495752960 00000100:00100000:1.0:1789760718.403426:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702634982738 time 702676985773 state 0x0 ref 1 00000100:00100000:1.0:1789760718.403441:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702677246189 time 702676985773 state 0x1 ref 1 00000100:00100000:1.0:1789760718.403463:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702677246189 time 702724180003 state 0x0 ref 1 00000100:00100000:1.0:1789760718.403475:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495752960:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760718.403527:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21573632, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa807 00010000:00010000:1.0:1789760718.403537:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457d1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.403559:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760718.403563:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760718.403570:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.403586:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760718.403809:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972583 is committed 00002000:00010000:1.0:1789760718.450429:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21573632 -> 21577728 00010000:00010000:1.0:1789760718.450454:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.450469:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457d1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.450480:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.450489:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760718.450494:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.450503:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457d1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.450523:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457d1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.450531:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457d1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760718.450647:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495752960:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47168us (47326us total) trans 4294972584 rc 0/0 00000100:00100000:1.0:1789760718.450658:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5296 00000100:00100000:1.0:1789760718.450669:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702677246189 time 702724180003 state 0x2 ref 0 00000001:00080000:3.0:1789760718.454914:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972584 is committed 00000100:00100000:1.0:1789760718.458475:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760718.458789:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495753088 00000100:00100000:0.0:1789760718.458800:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495753088 00000100:00100000:0.0:1789760718.458855:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702677246189 time 702724180003 state 0x0 ref 1 00000100:00100000:0.0:1789760718.458870:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702724440419 time 702724180003 state 0x1 ref 1 00000100:00100000:0.0:1789760718.458934:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702724440419 time 702779650528 state 0x0 ref 1 00000100:00100000:0.0:1789760718.458947:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495753088:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760718.458995:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21577728, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa807 00010000:00010000:0.0:1789760718.459003:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457d8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.459020:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760718.459025:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760718.459031:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.459044:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760718.494894:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972585 is committed 00002000:00010000:0.0:1789760718.495075:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21577728 -> 21581824 00010000:00010000:0.0:1789760718.495090:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.495103:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457d8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.495111:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.495118:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94aa00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760718.495122:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.495130:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457d8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.495149:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457d8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.495156:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457d8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.495553:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495753088:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36602us (37080us total) trans 4294972585 rc 0/0 00000100:00100000:0.0:1789760718.495568:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5297 00000100:00100000:0.0:1789760718.495577:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702724440419 time 702779650528 state 0x2 ref 0 00000100:00100000:2.0:1789760718.499630:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760718.502322:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495753216 00000100:00100000:3.0:1789760718.502336:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495753216 00000100:00100000:3.0:1789760718.502396:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702724440419 time 702779650528 state 0x0 ref 1 00000100:00100000:3.0:1789760718.502412:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702779910944 time 702779650528 state 0x1 ref 1 00000100:00100000:3.0:1789760718.502448:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702779910944 time 702823165336 state 0x0 ref 1 00000100:00100000:3.0:1789760718.502461:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495753216:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760718.502507:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21581824, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa807 00010000:00010000:3.0:1789760718.502515:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545457df lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.502530:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760718.502534:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760718.502552:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545457df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.502566:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545457df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760718.593871:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972586 is committed 00002000:00010000:3.0:1789760718.593945:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21581824 -> 21585920 00010000:00010000:3.0:1789760718.593961:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545457df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.593975:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545457df lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.593984:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545457df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.593991:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760718.593996:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545457df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.594005:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545457df lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.594024:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545457df lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.594032:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545457df lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.594115:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495753216:12345-192.168.203.9@tcp:4:dd.0 Request processed in 91652us (94494us total) trans 4294972586 rc 0/0 00000100:00100000:3.0:1789760718.594126:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5298 00000100:00100000:3.0:1789760718.594136:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702779910944 time 702823165336 state 0x2 ref 0 00000100:00100000:0.0:1789760718.598665:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760718.598759:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495753344 00000100:00100000:1.0:1789760718.598769:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495753344 00000100:00100000:1.0:1789760718.598824:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702779910944 time 702823165336 state 0x0 ref 1 00000100:00100000:1.0:1789760718.598840:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702823425752 time 702823165336 state 0x1 ref 1 00000100:00100000:1.0:1789760718.598903:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702823425752 time 702919619074 state 0x0 ref 1 00000100:00100000:1.0:1789760718.598915:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495753344:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760718.598960:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21585920, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa807 00010000:00010000:1.0:1789760718.598968:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457e6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.598985:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760718.598989:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760718.598996:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.599010:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760718.628011:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972587 is committed 00002000:00010000:1.0:1789760718.628386:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21585920 -> 21590016 00010000:00010000:1.0:1789760718.628402:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.628415:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457e6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.628424:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.628431:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760718.628436:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.628444:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457e6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.628464:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457e6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.628471:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457e6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760718.628581:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495753344:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29663us (29920us total) trans 4294972587 rc 0/0 00000100:00100000:1.0:1789760718.628591:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5299 00000100:00100000:1.0:1789760718.628603:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702823425752 time 702919619074 state 0x2 ref 0 00000100:00100000:3.0:1789760718.635025:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760718.635074:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495753472 00000100:00100000:0.0:1789760718.635084:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495753472 00000100:00100000:0.0:1789760718.635144:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702823425752 time 702919619074 state 0x0 ref 1 00000100:00100000:0.0:1789760718.635159:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702919879490 time 702919619074 state 0x1 ref 1 00000100:00100000:0.0:1789760718.635193:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702919879490 time 702955910164 state 0x0 ref 1 00000100:00100000:0.0:1789760718.635206:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495753472:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760718.635265:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21590016, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa807 00010000:00010000:0.0:1789760718.635274:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457ed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.635331:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760718.635345:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760718.635350:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.635364:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.689150:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760718.708064:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972588 is committed 00002000:00010000:2.0:1789760718.708454:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21590016 -> 21594112 00002000:00010000:2.0:1789760718.708459:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760717 -> 1789760718 00002000:00010000:2.0:1789760718.708462:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760717 -> 1789760718 00010000:00010000:2.0:1789760718.708474:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.708487:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457ed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.708495:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.708503:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94aa00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760718.708509:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.708520:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457ed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.708541:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457ed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.708549:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545457ed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.708719:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495753472:12345-192.168.203.9@tcp:4:dd.0 Request processed in 73510us (73698us total) trans 4294972588 rc 0/0 00000100:00100000:2.0:1789760718.708730:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5300 00000100:00100000:2.0:1789760718.708741:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702919879490 time 702955910164 state 0x2 ref 0 00000100:00100000:3.0:1789760718.713063:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760718.713169:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495753600 00000100:00100000:0.0:1789760718.713180:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495753600 00000100:00100000:0.0:1789760718.713243:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702919879490 time 702955910164 state 0x0 ref 1 00000100:00100000:0.0:1789760718.713259:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702956170580 time 702955910164 state 0x1 ref 1 00000100:00100000:0.0:1789760718.713324:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702956170580 time 703034040822 state 0x0 ref 1 00000100:00100000:0.0:1789760718.713337:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495753600:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760718.713385:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21594112, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa807 00010000:00010000:0.0:1789760718.713394:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545457f4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.713408:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760718.713412:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760718.713418:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545457f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.713432:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545457f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760718.740200:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972589 is committed 00002000:00010000:1.0:1789760718.740358:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21594112 -> 21598208 00010000:00010000:1.0:1789760718.740375:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545457f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.740387:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545457f4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.740397:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545457f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.740405:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760718.740410:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545457f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.740419:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545457f4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.740440:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545457f4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.740454:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545457f4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760718.740547:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495753600:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27207us (27487us total) trans 4294972589 rc 0/0 00000100:00100000:1.0:1789760718.740557:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5301 00000100:00100000:1.0:1789760718.740567:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702956170580 time 703034040822 state 0x2 ref 0 00000100:00100000:0.0:1789760718.744156:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760718.744264:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495753728 00000100:00100000:1.0:1789760718.744274:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495753728 00000100:00100000:1.0:1789760718.744357:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 702956170580 time 703034040822 state 0x0 ref 1 00000100:00100000:1.0:1789760718.744373:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703034301238 time 703034040822 state 0x1 ref 1 00000100:00100000:1.0:1789760718.744395:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703034301238 time 703065112327 state 0x0 ref 1 00000100:00100000:1.0:1789760718.744406:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04292a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495753728:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760718.744455:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21598208, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa807 00010000:00010000:1.0:1789760718.744464:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457fb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.744482:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760718.744487:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760718.744492:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.744510:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760718.793754:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972590 is committed 00002000:00010000:1.0:1789760718.793896:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21598208 -> 21602304 00010000:00010000:1.0:1789760718.793913:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.793926:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457fb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.793936:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.793944:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760718.793949:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.793957:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.793976:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457fb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.793984:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545457fb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760718.794085:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04292a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495753728:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49676us (49936us total) trans 4294972590 rc 0/0 00000100:00100000:1.0:1789760718.794096:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5302 00000100:00100000:1.0:1789760718.794106:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703034301238 time 703065112327 state 0x2 ref 0 00000100:00100000:3.0:1789760718.800140:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760718.800385:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495753856 00000100:00100000:2.0:1789760718.800395:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495753856 00000100:00100000:2.0:1789760718.800451:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703034301238 time 703065112327 state 0x0 ref 1 00000100:00100000:2.0:1789760718.800466:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703065372743 time 703065112327 state 0x1 ref 1 00000100:00100000:2.0:1789760718.802563:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703065372743 time 703123268569 state 0x0 ref 1 00000100:00100000:2.0:1789760718.802590:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495753856:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760718.802656:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21602304, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa807 00010000:00010000:2.0:1789760718.802666:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545802 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.802684:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760718.802690:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760718.802697:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545802 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.802712:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545802 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760718.849341:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972591 is committed 00002000:00010000:3.0:1789760718.849536:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21602304 -> 21606400 00010000:00010000:3.0:1789760718.849552:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545802 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.849566:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545802 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.849575:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545802 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.849582:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760718.849586:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545802 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.849595:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545802 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.849613:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545802 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.849622:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545802 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.849706:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495753856:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47118us (49566us total) trans 4294972591 rc 0/0 00000100:00100000:3.0:1789760718.849717:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5303 00000100:00100000:3.0:1789760718.849726:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703065372743 time 703123268569 state 0x2 ref 0 00000100:00100000:2.0:1789760718.852016:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760718.852133:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495753984 00000100:00100000:1.0:1789760718.852142:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495753984 00000100:00100000:1.0:1789760718.852259:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703065372743 time 703123268569 state 0x0 ref 1 00000100:00100000:1.0:1789760718.852353:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703123528985 time 703123268569 state 0x1 ref 1 00000100:00100000:1.0:1789760718.852378:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703123528985 time 703173094247 state 0x0 ref 1 00000100:00100000:1.0:1789760718.852391:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495753984:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760718.852441:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21606400, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa807 00010000:00010000:1.0:1789760718.852450:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545809 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.852466:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760718.852471:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760718.852477:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545809 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.852490:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545809 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760718.871010:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972592 is committed 00002000:00010000:1.0:1789760718.871181:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21606400 -> 21610496 00010000:00010000:1.0:1789760718.871204:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545809 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.871225:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545809 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.871234:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545809 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.871242:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760718.871247:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545809 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.871255:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545809 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.871273:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545809 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.871313:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545809 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760718.871403:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495753984:12345-192.168.203.9@tcp:4:dd.0 Request processed in 19010us (19392us total) trans 4294972592 rc 0/0 00000100:00100000:1.0:1789760718.871416:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5304 00000100:00100000:1.0:1789760718.871425:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703123528985 time 703173094247 state 0x2 ref 0 00000100:00100000:0.0:1789760718.875806:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760718.875885:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495754112 00000100:00100000:1.0:1789760718.875895:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495754112 00000100:00100000:1.0:1789760718.875950:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703123528985 time 703173094247 state 0x0 ref 1 00000100:00100000:1.0:1789760718.875965:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703173354663 time 703173094247 state 0x1 ref 1 00000100:00100000:1.0:1789760718.875997:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703173354663 time 703196713760 state 0x0 ref 1 00000100:00100000:1.0:1789760718.876009:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04290700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495754112:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760718.876053:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21610496, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa807 00010000:00010000:1.0:1789760718.876061:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354545810 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.876076:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760718.876080:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760718.876086:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354545810 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.876101:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354545810 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760718.939812:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972593 is committed 00002000:00010000:3.0:1789760718.939959:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21610496 -> 21614592 00010000:00010000:3.0:1789760718.939976:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354545810 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.939988:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354545810 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.939996:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354545810 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.940003:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760718.940007:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354545810 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.940015:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354545810 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.940035:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354545810 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.940043:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354545810 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.940136:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04290700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495754112:12345-192.168.203.9@tcp:4:dd.0 Request processed in 64124us (64334us total) trans 4294972593 rc 0/0 00000100:00100000:3.0:1789760718.940146:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5305 00000100:00100000:3.0:1789760718.940156:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703173354663 time 703196713760 state 0x2 ref 0 00000100:00100000:3.0:1789760718.944055:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760718.944207:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495754240 00000100:00100000:2.0:1789760718.944225:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495754240 00000100:00100000:2.0:1789760718.944320:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703173354663 time 703196713760 state 0x0 ref 1 00000100:00100000:2.0:1789760718.944335:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703196974176 time 703196713760 state 0x1 ref 1 00000100:00100000:2.0:1789760718.944355:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703196974176 time 703265072098 state 0x0 ref 1 00000100:00100000:2.0:1789760718.944367:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495754240:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760718.944412:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21614592, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa807 00010000:00010000:2.0:1789760718.944422:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545817 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.944439:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760718.944443:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760718.944449:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545817 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.944461:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545817 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760718.984264:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21614592 -> 21618688 00010000:00010000:2.0:1789760718.984392:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545817 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.984406:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545817 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.984415:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545817 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.984422:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760718.984426:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545817 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.984435:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545817 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.984455:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545817 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.984463:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545817 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.984657:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495754240:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40287us (40606us total) trans 4294972594 rc 0/0 00000100:00100000:2.0:1789760718.984668:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5306 00000100:00100000:2.0:1789760718.984678:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703196974176 time 703265072098 state 0x2 ref 0 00000001:00080000:2.0:1789760718.984722:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972594 is committed 00000100:00100000:2.0:1789760718.991546:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760718.991634:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495754368 00000100:00100000:1.0:1789760718.991644:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495754368 00000100:00100000:1.0:1789760718.991708:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703196974176 time 703265072098 state 0x0 ref 1 00000100:00100000:1.0:1789760718.991723:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703265332514 time 703265072098 state 0x1 ref 1 00000100:00100000:1.0:1789760718.991751:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703265332514 time 703312468115 state 0x0 ref 1 00000100:00100000:1.0:1789760718.991763:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495754368:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760718.991811:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21618688, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa807 00010000:00010000:1.0:1789760718.991820:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454581e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.991836:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760718.991841:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760718.991848:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454581e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.991861:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454581e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760719.047368:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972595 is committed 00002000:00010000:1.0:1789760719.047607:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21618688 -> 21622784 00010000:00010000:1.0:1789760719.047624:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454581e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.047637:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454581e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.047647:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454581e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.047655:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760719.047660:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454581e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.047670:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454581e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.047690:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454581e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.047697:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454581e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.047793:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495754368:12345-192.168.203.9@tcp:4:dd.0 Request processed in 56027us (56263us total) trans 4294972595 rc 0/0 00000100:00100000:1.0:1789760719.047804:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5307 00000100:00100000:1.0:1789760719.047813:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703265332514 time 703312468115 state 0x2 ref 0 00000100:00100000:2.0:1789760719.051912:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760719.052108:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495754496 00000100:00100000:3.0:1789760719.052119:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495754496 00000100:00100000:3.0:1789760719.052237:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703265332514 time 703312468115 state 0x0 ref 1 00000100:00100000:3.0:1789760719.052253:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703312728531 time 703312468115 state 0x1 ref 1 00000100:00100000:3.0:1789760719.052304:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703312728531 time 703373020366 state 0x0 ref 1 00000100:00100000:3.0:1789760719.052317:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495754496:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760719.052364:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21622784, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa807 00010000:00010000:3.0:1789760719.052373:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545825 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.052387:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760719.052391:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760719.052397:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545825 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.052409:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545825 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760719.053226:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789760719.114011:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21622784 -> 21626880 00010000:00010000:3.0:1789760719.114166:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545825 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.114195:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545825 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.114205:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545825 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.114221:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760719.114226:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545825 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.114235:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545825 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.114256:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545825 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.114263:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545825 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760719.114382:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495754496:12345-192.168.203.9@tcp:4:dd.0 Request processed in 62062us (62473us total) trans 4294972596 rc 0/0 00000100:00100000:3.0:1789760719.114393:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5308 00000100:00100000:3.0:1789760719.114403:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703312728531 time 703373020366 state 0x2 ref 0 00000100:00100000:3.0:1789760719.118545:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760719.118781:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495754624 00000100:00100000:0.0:1789760719.118791:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495754624 00000100:00100000:0.0:1789760719.118846:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703312728531 time 703373020366 state 0x0 ref 1 00000100:00100000:0.0:1789760719.118861:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703373280782 time 703373020366 state 0x1 ref 1 00000100:00100000:0.0:1789760719.118894:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703373280782 time 703439610198 state 0x0 ref 1 00000100:00100000:0.0:1789760719.118907:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495754624:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760719.118951:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21626880, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa807 00010000:00010000:0.0:1789760719.118960:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454582c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.118975:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760719.118979:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760719.118985:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454582c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.118999:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454582c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760719.129731:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972596 is committed 00000001:00080000:3.0:1789760719.175038:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972597 is committed 00002000:00010000:0.0:1789760719.175273:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21626880 -> 21630976 00010000:00010000:0.0:1789760719.175324:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454582c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.175337:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454582c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.175347:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454582c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.175355:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760719.175360:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454582c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.175369:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454582c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.175389:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454582c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.175397:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454582c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760719.175492:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495754624:12345-192.168.203.9@tcp:4:dd.0 Request processed in 56582us (56961us total) trans 4294972597 rc 0/0 00000100:00100000:0.0:1789760719.175503:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5309 00000100:00100000:0.0:1789760719.175511:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703373280782 time 703439610198 state 0x2 ref 0 00000100:00100000:0.0:1789760719.182589:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760719.182688:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495754752 00000100:00100000:2.0:1789760719.182697:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495754752 00000100:00100000:2.0:1789760719.182753:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703373280782 time 703439610198 state 0x0 ref 1 00000100:00100000:2.0:1789760719.182768:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703439870614 time 703439610198 state 0x1 ref 1 00000100:00100000:2.0:1789760719.182798:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703439870614 time 703503515242 state 0x0 ref 1 00000100:00100000:2.0:1789760719.182810:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495754752:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760719.182857:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21630976, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa807 00010000:00010000:2.0:1789760719.182865:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545833 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.182885:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760719.182890:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760719.182896:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545833 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.182909:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545833 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760719.238652:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972598 is committed 00002000:00010000:2.0:1789760719.238923:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21630976 -> 21635072 00010000:00010000:2.0:1789760719.238939:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545833 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.238951:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545833 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.238961:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545833 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.238968:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760719.238973:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545833 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.238982:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545833 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.239012:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545833 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.239020:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545833 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760719.239183:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495754752:12345-192.168.203.9@tcp:4:dd.0 Request processed in 56370us (56599us total) trans 4294972598 rc 0/0 00000100:00100000:2.0:1789760719.241692:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5310 00000100:00100000:2.0:1789760719.241706:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703439870614 time 703503515242 state 0x2 ref 0 00000100:00100000:2.0:1789760719.243792:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760719.243883:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495754880 00000100:00100000:3.0:1789760719.243893:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495754880 00000100:00100000:3.0:1789760719.243944:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703439870614 time 703503515242 state 0x0 ref 1 00000100:00100000:3.0:1789760719.243959:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703503775658 time 703503515242 state 0x1 ref 1 00000100:00100000:3.0:1789760719.244142:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703503775658 time 703564857530 state 0x0 ref 1 00000100:00100000:3.0:1789760719.244157:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495754880:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760719.244313:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21635072, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa807 00010000:00010000:3.0:1789760719.244322:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454583a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.244340:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760719.244345:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760719.244351:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454583a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.244367:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454583a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760719.278400:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972599 is committed 00002000:00010000:3.0:1789760719.278722:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21635072 -> 21639168 00010000:00010000:3.0:1789760719.278738:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454583a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.278757:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454583a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.278766:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454583a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.278773:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760719.278778:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454583a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.278785:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454583a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.278805:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454583a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.278812:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454583a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760719.278901:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495754880:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34742us (35109us total) trans 4294972599 rc 0/0 00000100:00100000:3.0:1789760719.278911:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5311 00000100:00100000:3.0:1789760719.278921:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703503775658 time 703564857530 state 0x2 ref 0 00000100:00100000:0.0:1789760719.283520:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760719.283725:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495755008 00000100:00100000:2.0:1789760719.283736:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495755008 00000100:00100000:2.0:1789760719.283792:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703503775658 time 703564857530 state 0x0 ref 1 00000100:00100000:2.0:1789760719.283807:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703565117946 time 703564857530 state 0x1 ref 1 00000100:00100000:2.0:1789760719.283838:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703565117946 time 703604554661 state 0x0 ref 1 00000100:00100000:2.0:1789760719.283850:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04291180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495755008:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760719.283904:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21639168, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa807 00010000:00010000:2.0:1789760719.283912:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545841 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.283929:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760719.283933:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760719.283940:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545841 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.283955:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545841 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760719.338760:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21639168 -> 21643264 00010000:00010000:0.0:1789760719.338780:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545841 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.338793:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545841 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.338801:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545841 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.338808:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760719.338813:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545841 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.338822:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545841 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.338841:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545841 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.338849:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545841 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760719.338939:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04291180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495755008:12345-192.168.203.9@tcp:4:dd.0 Request processed in 55087us (55424us total) trans 4294972600 rc 0/0 00000100:00100000:0.0:1789760719.338949:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5312 00000100:00100000:0.0:1789760719.338958:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703565117946 time 703604554661 state 0x2 ref 0 00000001:00080000:1.0:1789760719.339123:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972600 is committed 00000100:00100000:0.0:1789760719.342767:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760719.342842:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495755136 00000100:00100000:3.0:1789760719.342853:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495755136 00000100:00100000:3.0:1789760719.342904:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703565117946 time 703604554661 state 0x0 ref 1 00000100:00100000:3.0:1789760719.342919:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703604815077 time 703604554661 state 0x1 ref 1 00000100:00100000:3.0:1789760719.342949:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703604815077 time 703663666388 state 0x0 ref 1 00000100:00100000:3.0:1789760719.342962:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495755136:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760719.343008:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21643264, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa807 00010000:00010000:3.0:1789760719.343016:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545848 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.343104:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760719.343110:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760719.343117:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545848 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.343133:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545848 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760719.401120:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972601 is committed 00002000:00010000:3.0:1789760719.402161:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21643264 -> 21647360 00010000:00010000:3.0:1789760719.402180:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545848 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.402269:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545848 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.402378:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545848 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.402386:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760719.402391:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545848 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.402399:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545848 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.402419:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545848 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.402427:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545848 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760719.402615:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495755136:12345-192.168.203.9@tcp:4:dd.0 Request processed in 59649us (59852us total) trans 4294972601 rc 0/0 00000100:00100000:2.0:1789760719.402627:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5313 00000100:00100000:2.0:1789760719.402638:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703604815077 time 703663666388 state 0x2 ref 0 00000100:00100000:2.0:1789760719.406888:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760719.406987:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495755264 00000100:00100000:3.0:1789760719.406997:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495755264 00000100:00100000:3.0:1789760719.407049:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703604815077 time 703663666388 state 0x0 ref 1 00000100:00100000:3.0:1789760719.407064:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703663926804 time 703663666388 state 0x1 ref 1 00000100:00100000:3.0:1789760719.407143:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703663926804 time 703727858791 state 0x0 ref 1 00000100:00100000:3.0:1789760719.407157:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495755264:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760719.407228:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21647360, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa807 00010000:00010000:3.0:1789760719.407237:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454584f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.407253:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760719.407257:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760719.407264:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454584f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.407318:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454584f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760719.439803:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972602 is committed 00002000:00010000:2.0:1789760719.439994:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21647360 -> 21651456 00010000:00010000:2.0:1789760719.440010:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454584f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.440021:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454584f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.440030:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454584f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.440038:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760719.440042:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454584f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.440050:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454584f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.440069:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454584f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.440077:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454584f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760719.440218:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495755264:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33050us (33324us total) trans 4294972602 rc 0/0 00000100:00100000:2.0:1789760719.440229:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5314 00000100:00100000:2.0:1789760719.440239:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703663926804 time 703727858791 state 0x2 ref 0 00000100:00100000:0.0:1789760719.444757:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760719.444838:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495755392 00000100:00100000:2.0:1789760719.444848:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495755392 00000100:00100000:2.0:1789760719.444911:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703663926804 time 703727858791 state 0x0 ref 1 00000100:00100000:2.0:1789760719.444927:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703728119207 time 703727858791 state 0x1 ref 1 00000100:00100000:2.0:1789760719.444952:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703728119207 time 703765669354 state 0x0 ref 1 00000100:00100000:2.0:1789760719.444965:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495755392:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760719.445012:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21651456, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa807 00010000:00010000:2.0:1789760719.445022:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545856 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.445037:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760719.445043:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760719.445049:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545856 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.445063:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545856 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760719.487165:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972603 is committed 00002000:00010000:2.0:1789760719.487425:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21651456 -> 21655552 00010000:00010000:2.0:1789760719.487441:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545856 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.487455:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545856 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.487463:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545856 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.487470:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760719.487474:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545856 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.487483:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545856 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.487502:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545856 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.487514:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545856 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760719.487611:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495755392:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42634us (42847us total) trans 4294972603 rc 0/0 00000100:00100000:2.0:1789760719.487622:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5315 00000100:00100000:2.0:1789760719.487632:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703728119207 time 703765669354 state 0x2 ref 0 00000100:00100000:1.0:1789760719.491564:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760719.491841:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495755520 00000100:00100000:2.0:1789760719.491852:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495755520 00000100:00100000:2.0:1789760719.491902:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703728119207 time 703765669354 state 0x0 ref 1 00000100:00100000:2.0:1789760719.491916:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703765929770 time 703765669354 state 0x1 ref 1 00000100:00100000:2.0:1789760719.491986:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703765929770 time 703812701491 state 0x0 ref 1 00000100:00100000:2.0:1789760719.491998:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495755520:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760719.492047:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21655552, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa807 00010000:00010000:2.0:1789760719.492056:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454585d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.492072:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760719.492076:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760719.492082:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454585d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.492099:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454585d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760719.541011:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972604 is committed 00002000:00010000:2.0:1789760719.541136:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21655552 -> 21659648 00010000:00010000:2.0:1789760719.541151:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454585d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.541164:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454585d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.541173:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454585d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.541180:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760719.541193:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454585d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.541202:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454585d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.541231:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454585d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.541239:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454585d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760719.541365:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495755520:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49364us (49806us total) trans 4294972604 rc 0/0 00000100:00100000:2.0:1789760719.541376:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5316 00000100:00100000:2.0:1789760719.541385:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703765929770 time 703812701491 state 0x2 ref 0 00000100:00100000:2.0:1789760719.544999:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760719.545136:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495755648 00000100:00100000:3.0:1789760719.545146:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495755648 00000100:00100000:3.0:1789760719.545966:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703765929770 time 703812701491 state 0x0 ref 1 00000100:00100000:3.0:1789760719.545986:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703812961907 time 703812701491 state 0x1 ref 1 00000100:00100000:3.0:1789760719.546021:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703812961907 time 703866737286 state 0x0 ref 1 00000100:00100000:3.0:1789760719.546035:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495755648:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760719.546081:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21659648, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa807 00010000:00010000:3.0:1789760719.546090:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545864 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.546107:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760719.546112:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760719.546119:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545864 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.546133:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545864 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760719.621527:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972605 is committed 00002000:00010000:3.0:1789760719.621731:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21659648 -> 21663744 00010000:00010000:3.0:1789760719.621747:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545864 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.621761:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545864 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.621769:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545864 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.621776:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760719.621780:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545864 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.621788:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545864 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.621807:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545864 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.621818:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545864 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760719.621976:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495755648:12345-192.168.203.9@tcp:4:dd.0 Request processed in 75939us (76980us total) trans 4294972605 rc 0/0 00000100:00100000:3.0:1789760719.621988:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5317 00000100:00100000:3.0:1789760719.621998:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703812961907 time 703866737286 state 0x2 ref 0 00000100:00100000:2.0:1789760719.630276:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760719.630395:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495755776 00000100:00100000:0.0:1789760719.630405:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495755776 00000100:00100000:0.0:1789760719.630457:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703812961907 time 703866737286 state 0x0 ref 1 00000100:00100000:0.0:1789760719.630472:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703866997702 time 703866737286 state 0x1 ref 1 00000100:00100000:0.0:1789760719.630509:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703866997702 time 703951225786 state 0x0 ref 1 00000100:00100000:0.0:1789760719.630522:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495755776:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760719.630568:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21663744, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa807 00010000:00010000:0.0:1789760719.630577:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454586b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.630594:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760719.630598:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760719.630604:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454586b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.630616:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454586b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760719.706474:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972606 is committed 00002000:00010000:1.0:1789760719.706598:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21663744 -> 21667840 00002000:00010000:1.0:1789760719.706603:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760718 -> 1789760719 00002000:00010000:1.0:1789760719.706606:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760718 -> 1789760719 00010000:00010000:1.0:1789760719.706620:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454586b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.706633:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454586b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.706643:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454586b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.706651:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760719.706656:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454586b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.706666:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454586b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.706689:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454586b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.706698:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454586b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.706796:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495755776:12345-192.168.203.9@tcp:4:dd.0 Request processed in 76271us (76525us total) trans 4294972606 rc 0/0 00000100:00100000:1.0:1789760719.706812:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5318 00000100:00100000:1.0:1789760719.706822:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703866997702 time 703951225786 state 0x2 ref 0 00000100:00100000:1.0:1789760719.711220:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760719.711323:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495755904 00000100:00100000:2.0:1789760719.711333:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495755904 00000100:00100000:2.0:1789760719.711386:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703866997702 time 703951225786 state 0x0 ref 1 00000100:00100000:2.0:1789760719.711401:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703951486202 time 703951225786 state 0x1 ref 1 00000100:00100000:2.0:1789760719.711426:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703951486202 time 704032143132 state 0x0 ref 1 00000100:00100000:2.0:1789760719.711437:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495755904:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760719.711486:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21667840, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xa807 00010000:00010000:2.0:1789760719.711495:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545872 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.711510:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760719.711515:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760719.711522:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545872 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.711541:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545872 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.712228:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760719.774838:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972607 is committed 00002000:00010000:0.0:1789760719.775414:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21667840 -> 21671936 00010000:00010000:0.0:1789760719.775429:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545872 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.775443:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545872 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.775452:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545872 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.775464:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760719.775469:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545872 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.775478:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545872 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.775819:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545872 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.775828:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545872 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760719.776246:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495755904:12345-192.168.203.9@tcp:4:dd.0 Request processed in 64479us (64714us total) trans 4294972607 rc 0/0 00000100:00100000:0.0:1789760719.776257:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5319 00000100:00100000:0.0:1789760719.776267:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703951486202 time 704032143132 state 0x2 ref 0 00000100:00100000:0.0:1789760719.783807:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760719.783898:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495756032 00000100:00100000:3.0:1789760719.783908:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495756032 00000100:00100000:3.0:1789760719.783961:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 703951486202 time 704032143132 state 0x0 ref 1 00000100:00100000:3.0:1789760719.783976:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704032403548 time 704032143132 state 0x1 ref 1 00000100:00100000:3.0:1789760719.784012:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704032403548 time 704104728675 state 0x0 ref 1 00000100:00100000:3.0:1789760719.784026:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495756032:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760719.784071:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21671936, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xa807 00010000:00010000:3.0:1789760719.784081:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545879 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.784098:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760719.784103:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760719.784111:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545879 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.784125:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545879 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760719.845131:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972608 is committed 00002000:00010000:3.0:1789760719.845212:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21671936 -> 21676032 00010000:00010000:3.0:1789760719.845227:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545879 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.845250:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545879 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.845260:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545879 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.845267:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760719.845272:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545879 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.845316:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545879 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.845335:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545879 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.845343:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545879 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760719.845433:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495756032:12345-192.168.203.9@tcp:4:dd.0 Request processed in 61405us (61629us total) trans 4294972608 rc 0/0 00000100:00100000:3.0:1789760719.845443:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5320 00000100:00100000:3.0:1789760719.845452:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704032403548 time 704104728675 state 0x2 ref 0 00000100:00100000:1.0:1789760719.859155:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760719.859268:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495756160 00000100:00100000:0.0:1789760719.859313:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495756160 00000100:00100000:0.0:1789760719.859379:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704032403548 time 704104728675 state 0x0 ref 1 00000100:00100000:0.0:1789760719.859394:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704104989091 time 704104728675 state 0x1 ref 1 00000100:00100000:0.0:1789760719.859426:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704104989091 time 704180142901 state 0x0 ref 1 00000100:00100000:0.0:1789760719.859440:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495756160:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760719.859487:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21676032, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xa807 00010000:00010000:0.0:1789760719.859496:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545880 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.859511:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760719.859516:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760719.859522:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545880 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.859536:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545880 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760719.901869:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21676032 -> 21680128 00010000:00010000:1.0:1789760719.901891:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545880 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.901905:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545880 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.901914:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545880 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.901921:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760719.901927:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545880 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.901937:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545880 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.901958:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545880 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.901966:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545880 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.902081:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495756160:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42639us (42931us total) trans 4294972609 rc 0/0 00000100:00100000:1.0:1789760719.902093:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5321 00000100:00100000:1.0:1789760719.902103:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704104989091 time 704180142901 state 0x2 ref 0 00000001:00080000:0.0:1789760719.903737:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972609 is committed 00000100:00100000:1.0:1789760719.906435:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760719.906597:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495756288 00000100:00100000:2.0:1789760719.906606:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495756288 00000100:00100000:2.0:1789760719.906661:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704104989091 time 704180142901 state 0x0 ref 1 00000100:00100000:2.0:1789760719.906675:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704180403317 time 704180142901 state 0x1 ref 1 00000100:00100000:2.0:1789760719.906715:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704180403317 time 704227431811 state 0x0 ref 1 00000100:00100000:2.0:1789760719.906727:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495756288:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760719.906777:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21680128, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xa807 00010000:00010000:2.0:1789760719.906786:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545887 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.906801:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760719.906805:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760719.906811:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545887 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.906826:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545887 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760719.957176:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972610 is committed 00002000:00010000:2.0:1789760719.957683:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21680128 -> 21684224 00010000:00010000:2.0:1789760719.957699:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545887 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.957712:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545887 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.957721:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545887 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.957729:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760719.957735:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545887 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.957744:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545887 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.957760:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545887 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.957768:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545887 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760719.957857:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495756288:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51127us (51427us total) trans 4294972610 rc 0/0 00000100:00100000:2.0:1789760719.957867:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5322 00000100:00100000:2.0:1789760719.957979:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704180403317 time 704227431811 state 0x2 ref 0 00000100:00100000:3.0:1789760719.967910:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760719.968519:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495756416 00000100:00100000:0.0:1789760719.968531:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495756416 00000100:00100000:0.0:1789760719.968587:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704180403317 time 704227431811 state 0x0 ref 1 00000100:00100000:0.0:1789760719.968602:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704227692227 time 704227431811 state 0x1 ref 1 00000100:00100000:0.0:1789760719.968631:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704227692227 time 704289347147 state 0x0 ref 1 00000100:00100000:0.0:1789760719.968644:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495756416:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760719.968701:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21684224, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xa807 00010000:00010000:0.0:1789760719.968710:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454588e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.968726:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760719.968731:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760719.968738:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454588e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.968753:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454588e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760720.076828:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972611 is committed 00002000:00010000:0.0:1789760720.077001:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21684224 -> 21688320 00010000:00010000:0.0:1789760720.077017:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454588e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.077030:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454588e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.077041:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454588e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.077049:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760720.077055:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454588e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.077065:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454588e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.077084:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454588e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.077094:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454588e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760720.077210:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495756416:12345-192.168.203.9@tcp:4:dd.0 Request processed in 108552us (109293us total) trans 4294972611 rc 0/0 00000100:00100000:0.0:1789760720.077220:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5323 00000100:00100000:0.0:1789760720.077230:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704227692227 time 704289347147 state 0x2 ref 0 00000100:00100000:0.0:1789760720.081964:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760720.082051:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495756544 00000100:00100000:1.0:1789760720.082060:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495756544 00000100:00100000:1.0:1789760720.082115:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704227692227 time 704289347147 state 0x0 ref 1 00000100:00100000:1.0:1789760720.082130:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704289607563 time 704289347147 state 0x1 ref 1 00000100:00100000:1.0:1789760720.082158:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704289607563 time 704402874545 state 0x0 ref 1 00000100:00100000:1.0:1789760720.082170:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495756544:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760720.082448:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21688320, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xa807 00010000:00010000:1.0:1789760720.082459:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545895 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.082476:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760720.082481:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.082626:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545895 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.082652:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545895 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760720.161479:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789760720.161506:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff92900fab1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520302848:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760720.161585:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760720.167450:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520302848 00000100:00100000:2.0:1789760720.167465:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520302848 00000100:00100000:2.0:1789760720.167539:0:6711:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 191 00000100:00100000:2.0:1789760720.167550:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520302848:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760720.167578:0:6711:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527761408 pending 1703936 free 3836739584 avail 3810525184 00000100:00100000:2.0:1789760720.167659:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520302848:12345-0@lo:13:osp-pre-0-0.0 Request processed in 110us (6073us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760720.167666:0:6711:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 191 00000100:00100000:2.0:1789760720.167720:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff92900fab1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520302848:0@lo:13:osp-pre-0-0.0 00000100:00080000:1.0:1789760720.167786:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760720.175296:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972612 is committed 00002000:00010000:3.0:1789760720.175539:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21688320 -> 21692416 00010000:00010000:3.0:1789760720.175553:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545895 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.175567:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545895 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.175576:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545895 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.175584:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760720.175590:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545895 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.175598:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545895 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.175620:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545895 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.175628:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354545895 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760720.175723:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495756544:12345-192.168.203.9@tcp:4:dd.0 Request processed in 93549us (93762us total) trans 4294972612 rc 0/0 00000100:00100000:3.0:1789760720.175734:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5324 00000100:00100000:3.0:1789760720.175745:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704289607563 time 704402874545 state 0x2 ref 0 00000100:00100000:1.0:1789760720.180658:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760720.180755:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495756672 00000100:00100000:2.0:1789760720.180773:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495756672 00000100:00100000:2.0:1789760720.180827:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704289607563 time 704402874545 state 0x0 ref 1 00000100:00100000:2.0:1789760720.180842:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704403134961 time 704402874545 state 0x1 ref 1 00000100:00100000:2.0:1789760720.180873:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704403134961 time 704501589540 state 0x0 ref 1 00000100:00100000:2.0:1789760720.180884:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495756672:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760720.180933:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21692416, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xa807 00010000:00010000:2.0:1789760720.180942:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454589c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.180963:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760720.180966:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760720.180972:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454589c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.180985:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454589c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760720.265156:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972613 is committed 00002000:00010000:3.0:1789760720.266011:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21692416 -> 21696512 00010000:00010000:3.0:1789760720.266028:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454589c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.266041:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454589c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.266050:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454589c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.266058:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760720.266063:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454589c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.266073:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454589c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.266099:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454589c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.266106:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454589c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760720.266641:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495756672:12345-192.168.203.9@tcp:4:dd.0 Request processed in 85752us (85984us total) trans 4294972613 rc 0/0 00000100:00100000:3.0:1789760720.266652:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5325 00000100:00100000:3.0:1789760720.266662:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704403134961 time 704501589540 state 0x2 ref 0 00000100:00100000:0.0:1789760720.272752:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760720.273308:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495756800 00000100:00100000:2.0:1789760720.273319:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495756800 00000100:00100000:2.0:1789760720.273386:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704403134961 time 704501589540 state 0x0 ref 1 00000100:00100000:2.0:1789760720.273402:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704501849956 time 704501589540 state 0x1 ref 1 00000100:00100000:2.0:1789760720.273423:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704501849956 time 704594140392 state 0x0 ref 1 00000100:00100000:2.0:1789760720.273437:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903639cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495756800:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760720.273486:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21696512, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xa807 00010000:00010000:2.0:1789760720.273496:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545458a3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.273513:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760720.273518:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760720.273537:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545458a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.273558:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545458a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760720.346010:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972614 is committed 00002000:00010000:3.0:1789760720.346567:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21696512 -> 21700608 00010000:00010000:3.0:1789760720.346583:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545458a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.346595:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545458a3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.346604:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545458a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.346611:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760720.346616:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545458a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.346629:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545458a3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.346649:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545458a3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.346657:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545458a3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760720.346743:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903639cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495756800:12345-192.168.203.9@tcp:4:dd.0 Request processed in 73305us (73996us total) trans 4294972614 rc 0/0 00000100:00100000:3.0:1789760720.346752:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5326 00000100:00100000:3.0:1789760720.346770:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704501849956 time 704594140392 state 0x2 ref 0 00000100:00100000:0.0:1789760720.355862:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760720.356006:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495756928 00000100:00100000:3.0:1789760720.356016:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495756928 00000100:00100000:3.0:1789760720.356074:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704501849956 time 704594140392 state 0x0 ref 1 00000100:00100000:3.0:1789760720.356090:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704594400808 time 704594140392 state 0x1 ref 1 00000100:00100000:3.0:1789760720.356134:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704594400808 time 704676850586 state 0x0 ref 1 00000100:00100000:3.0:1789760720.356146:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495756928:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760720.356787:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21700608, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xa807 00010000:00010000:3.0:1789760720.356800:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545458aa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.356819:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760720.356823:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760720.356830:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545458aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.356845:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545458aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760720.423391:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972615 is committed 00002000:00010000:1.0:1789760720.423580:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21700608 -> 21704704 00010000:00010000:1.0:1789760720.423597:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545458aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.423609:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545458aa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.423618:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545458aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.423625:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760720.423630:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545458aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.423638:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545458aa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.423657:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545458aa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.423668:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545458aa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760720.423763:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495756928:12345-192.168.203.9@tcp:4:dd.0 Request processed in 67614us (67907us total) trans 4294972615 rc 0/0 00000100:00100000:1.0:1789760720.423774:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5327 00000100:00100000:1.0:1789760720.423785:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704594400808 time 704676850586 state 0x2 ref 0 00000100:00100000:1.0:1789760720.428146:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760720.428416:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495757056 00000100:00100000:3.0:1789760720.428426:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495757056 00000100:00100000:3.0:1789760720.428479:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704594400808 time 704676850586 state 0x0 ref 1 00000100:00100000:3.0:1789760720.428496:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704677111002 time 704676850586 state 0x1 ref 1 00000100:00100000:0.0:1789760720.428637:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704677111002 time 704749350821 state 0x0 ref 1 00000100:00100000:0.0:1789760720.428653:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495757056:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760720.428705:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21704704, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xa807 00010000:00010000:0.0:1789760720.428715:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545458b1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.428731:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760720.428736:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760720.428743:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545458b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.428762:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545458b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760720.521843:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972616 is committed 00002000:00010000:3.0:1789760720.522072:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21704704 -> 21708800 00010000:00010000:3.0:1789760720.522087:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545458b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.522170:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545458b1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.522181:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545458b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.522189:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760720.522194:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545458b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.522316:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545458b1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.522339:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545458b1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.522849:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545458b1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760720.522942:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495757056:12345-192.168.203.9@tcp:4:dd.0 Request processed in 94287us (94802us total) trans 4294972616 rc 0/0 00000100:00100000:3.0:1789760720.522953:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5328 00000100:00100000:3.0:1789760720.522963:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704677111002 time 704749350821 state 0x2 ref 0 00000100:00100000:0.0:1789760720.533045:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760720.533929:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495757184 00000100:00100000:2.0:1789760720.533942:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495757184 00000100:00100000:2.0:1789760720.534003:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704677111002 time 704749350821 state 0x0 ref 1 00000100:00100000:2.0:1789760720.534020:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704749611237 time 704749350821 state 0x1 ref 1 00000100:00100000:2.0:1789760720.534055:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704749611237 time 704854771767 state 0x0 ref 1 00000100:00100000:2.0:1789760720.534069:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495757184:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760720.534132:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21708800, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xa807 00010000:00010000:2.0:1789760720.534142:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545458b8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.534162:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760720.534167:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760720.534185:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545458b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.534214:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545458b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760720.610504:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972617 is committed 00002000:00010000:3.0:1789760720.610675:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21708800 -> 21712896 00010000:00010000:3.0:1789760720.610689:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545458b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.610701:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545458b8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.610710:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545458b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.610716:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760720.610722:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545458b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.610731:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545458b8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.610749:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545458b8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.610756:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545458b8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760720.610844:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495757184:12345-192.168.203.9@tcp:4:dd.0 Request processed in 76773us (77803us total) trans 4294972617 rc 0/0 00000100:00100000:3.0:1789760720.610854:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5329 00000100:00100000:3.0:1789760720.610864:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704749611237 time 704854771767 state 0x2 ref 0 00000100:00100000:0.0:1789760720.619825:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760720.619929:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495757312 00000100:00100000:1.0:1789760720.619939:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495757312 00000100:00100000:1.0:1789760720.619989:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704749611237 time 704854771767 state 0x0 ref 1 00000100:00100000:1.0:1789760720.620005:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704855032183 time 704854771767 state 0x1 ref 1 00000100:00100000:1.0:1789760720.620032:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704855032183 time 704940749086 state 0x0 ref 1 00000100:00100000:1.0:1789760720.620043:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495757312:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760720.620087:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21712896, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xa807 00010000:00010000:1.0:1789760720.620096:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545458bf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.620111:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760720.620115:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.620121:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545458bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.620134:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545458bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760720.666791:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972618 is committed 00002000:00010000:1.0:1789760720.666896:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21712896 -> 21716992 00002000:00010000:1.0:1789760720.666901:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760719 -> 1789760720 00002000:00010000:1.0:1789760720.666903:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760719 -> 1789760720 00010000:00010000:1.0:1789760720.666917:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545458bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.666929:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545458bf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.666938:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545458bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.666945:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760720.666950:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545458bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.666959:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545458bf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.666978:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545458bf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.666985:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545458bf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760720.667078:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495757312:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47030us (47257us total) trans 4294972618 rc 0/0 00000100:00100000:1.0:1789760720.667088:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5330 00000100:00100000:1.0:1789760720.667097:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704855032183 time 704940749086 state 0x2 ref 0 00000100:00100000:1.0:1789760720.671781:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760720.671864:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495757440 00000100:00100000:3.0:1789760720.671873:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495757440 00000100:00100000:3.0:1789760720.671927:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704855032183 time 704940749086 state 0x0 ref 1 00000100:00100000:3.0:1789760720.671943:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704941009502 time 704940749086 state 0x1 ref 1 00000100:00100000:3.0:1789760720.672023:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704941009502 time 704992738474 state 0x0 ref 1 00000100:00100000:3.0:1789760720.672037:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495757440:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760720.672084:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21716992, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa807 00010000:00010000:3.0:1789760720.672093:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545458c6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.672109:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760720.672114:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760720.672121:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545458c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.672135:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545458c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760720.706576:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972619 is committed 00002000:00010000:3.0:1789760720.706769:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21716992 -> 21721088 00010000:00010000:3.0:1789760720.706784:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545458c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.706797:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545458c6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.706806:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545458c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.706813:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760720.706818:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545458c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.706826:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545458c6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.706853:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545458c6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.706860:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545458c6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760720.706952:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495757440:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34912us (35174us total) trans 4294972619 rc 0/0 00000100:00100000:3.0:1789760720.706961:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5331 00000100:00100000:3.0:1789760720.706971:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704941009502 time 704992738474 state 0x2 ref 0 00000100:00100000:3.0:1789760720.711075:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760720.711167:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495757568 00000100:00100000:2.0:1789760720.711181:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495757568 00000100:00100000:2.0:1789760720.711334:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704941009502 time 704992738474 state 0x0 ref 1 00000100:00100000:2.0:1789760720.711349:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704992998890 time 704992738474 state 0x1 ref 1 00000100:00100000:2.0:1789760720.711372:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704992998890 time 705032089817 state 0x0 ref 1 00000100:00100000:2.0:1789760720.711384:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495757568:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760720.711430:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21721088, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa807 00010000:00010000:2.0:1789760720.711439:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545458cd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.711456:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760720.711461:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760720.711468:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545458cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.711484:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545458cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.734840:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760720.742966:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972620 is committed 00002000:00010000:3.0:1789760720.743068:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21721088 -> 21725184 00010000:00010000:3.0:1789760720.743085:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545458cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.743098:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545458cd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.743106:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545458cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.743114:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760720.743119:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545458cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.743133:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545458cd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.743153:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545458cd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.743160:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545458cd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760720.743488:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495757568:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32100us (32416us total) trans 4294972620 rc 0/0 00000100:00100000:3.0:1789760720.743499:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5332 00000100:00100000:3.0:1789760720.743509:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704992998890 time 705032089817 state 0x2 ref 0 00000100:00100000:2.0:1789760720.747148:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760720.747461:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495757696 00000100:00100000:1.0:1789760720.747471:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495757696 00000100:00100000:1.0:1789760720.747538:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 704992998890 time 705032089817 state 0x0 ref 1 00000100:00100000:1.0:1789760720.747553:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705032350233 time 705032089817 state 0x1 ref 1 00000100:00100000:1.0:1789760720.747585:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705032350233 time 705068300859 state 0x0 ref 1 00000100:00100000:1.0:1789760720.747597:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495757696:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760720.747647:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21725184, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa807 00010000:00010000:1.0:1789760720.747656:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458d4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.747672:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760720.747677:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.747684:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.747698:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760720.773705:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21725184 -> 21729280 00010000:00010000:1.0:1789760720.773724:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.773738:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458d4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.773746:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.773754:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760720.773759:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.773768:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458d4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.773787:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458d4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.773800:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458d4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760720.773893:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495757696:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26292us (26749us total) trans 4294972621 rc 0/0 00000100:00100000:1.0:1789760720.773905:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5333 00000100:00100000:1.0:1789760720.773914:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705032350233 time 705068300859 state 0x2 ref 0 00000001:00080000:1.0:1789760720.773954:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972621 is committed 00000100:00100000:1.0:1789760720.777923:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760720.777998:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495757824 00000100:00100000:3.0:1789760720.778007:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495757824 00000100:00100000:3.0:1789760720.778057:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705032350233 time 705068300859 state 0x0 ref 1 00000100:00100000:3.0:1789760720.778072:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705068561275 time 705068300859 state 0x1 ref 1 00000100:00100000:3.0:1789760720.778106:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705068561275 time 705098822299 state 0x0 ref 1 00000100:00100000:3.0:1789760720.778118:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495757824:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760720.778162:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21729280, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa807 00010000:00010000:3.0:1789760720.778171:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545458db lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.778209:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760720.778215:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760720.778221:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545458db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.778235:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545458db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760720.884923:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972622 is committed 00002000:00010000:1.0:1789760720.885126:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21729280 -> 21733376 00010000:00010000:1.0:1789760720.885141:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545458db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.885153:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545458db lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.885162:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545458db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.885170:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760720.885175:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545458db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.885184:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545458db lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.885215:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545458db lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.885224:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545458db lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760720.885349:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495757824:12345-192.168.203.9@tcp:4:dd.0 Request processed in 107227us (107429us total) trans 4294972622 rc 0/0 00000100:00100000:1.0:1789760720.885358:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5334 00000100:00100000:1.0:1789760720.885368:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705068561275 time 705098822299 state 0x2 ref 0 00000100:00100000:1.0:1789760720.895344:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760720.895537:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495757952 00000100:00100000:3.0:1789760720.895546:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495757952 00000100:00100000:3.0:1789760720.895598:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705068561275 time 705098822299 state 0x0 ref 1 00000100:00100000:3.0:1789760720.895613:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705099082715 time 705098822299 state 0x1 ref 1 00000100:00100000:3.0:1789760720.895648:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705099082715 time 705216364566 state 0x0 ref 1 00000100:00100000:3.0:1789760720.895660:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495757952:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760720.895704:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21733376, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa807 00010000:00010000:3.0:1789760720.895712:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545458e2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.895727:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760720.895731:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760720.895736:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545458e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.895765:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545458e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760720.939124:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21733376 -> 21737472 00010000:00010000:1.0:1789760720.939143:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545458e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.939157:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545458e2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.939166:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545458e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.939182:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760720.939187:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545458e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.939219:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545458e2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.939239:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545458e2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.939247:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545458e2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760720.939368:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495757952:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43705us (44029us total) trans 4294972623 rc 0/0 00000100:00100000:1.0:1789760720.939378:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5335 00000100:00100000:1.0:1789760720.939387:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705099082715 time 705216364566 state 0x2 ref 0 00000001:00080000:2.0:1789760720.939802:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972623 is committed 00000100:00100000:2.0:1789760720.947019:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760720.947361:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495758080 00000100:00100000:1.0:1789760720.947371:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495758080 00000100:00100000:1.0:1789760720.947423:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705099082715 time 705216364566 state 0x0 ref 1 00000100:00100000:1.0:1789760720.947437:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705216624982 time 705216364566 state 0x1 ref 1 00000100:00100000:1.0:1789760720.947465:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705216624982 time 705268182488 state 0x0 ref 1 00000100:00100000:1.0:1789760720.947477:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495758080:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760720.947531:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21737472, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa807 00010000:00010000:1.0:1789760720.947539:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458e9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.947554:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760720.947559:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.947565:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.947578:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760721.034093:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972624 is committed 00002000:00010000:1.0:1789760721.034776:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21737472 -> 21741568 00010000:00010000:1.0:1789760721.034793:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.034805:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458e9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.034814:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.034822:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760721.034827:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.034835:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458e9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.034855:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458e9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.034863:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458e9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.034964:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495758080:12345-192.168.203.9@tcp:4:dd.0 Request processed in 87483us (87950us total) trans 4294972624 rc 0/0 00000100:00100000:1.0:1789760721.034976:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5336 00000100:00100000:1.0:1789760721.034986:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705216624982 time 705268182488 state 0x2 ref 0 00000100:00100000:2.0:1789760721.041358:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760721.043716:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495758208 00000100:00100000:1.0:1789760721.043732:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495758208 00000100:00100000:1.0:1789760721.043822:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705216624982 time 705268182488 state 0x0 ref 1 00000100:00100000:1.0:1789760721.043839:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705268442904 time 705268182488 state 0x1 ref 1 00000100:00100000:1.0:1789760721.043872:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705268442904 time 705364589347 state 0x0 ref 1 00000100:00100000:1.0:1789760721.043894:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495758208:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760721.043945:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21741568, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa807 00010000:00010000:1.0:1789760721.043954:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545458f0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.043972:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760721.043977:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.043984:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545458f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.044003:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545458f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760721.046874:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760721.113735:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972625 is committed 00002000:00010000:2.0:1789760721.114002:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21741568 -> 21745664 00010000:00010000:2.0:1789760721.114021:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545458f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.114036:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545458f0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.114045:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545458f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.114053:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760721.114057:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545458f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.114068:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545458f0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.114091:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545458f0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.114099:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545458f0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760721.114274:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495758208:12345-192.168.203.9@tcp:4:dd.0 Request processed in 70386us (72919us total) trans 4294972625 rc 0/0 00000100:00100000:2.0:1789760721.114325:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5337 00000100:00100000:2.0:1789760721.114335:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705268442904 time 705364589347 state 0x2 ref 0 00000100:00100000:3.0:1789760721.118744:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760721.119132:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495758336 00000100:00100000:0.0:1789760721.119142:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495758336 00000100:00100000:0.0:1789760721.119273:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705268442904 time 705364589347 state 0x0 ref 1 00000100:00100000:0.0:1789760721.119400:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705364849763 time 705364589347 state 0x1 ref 1 00000100:00100000:0.0:1789760721.119465:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705364849763 time 705440180297 state 0x0 ref 1 00000100:00100000:0.0:1789760721.119480:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495758336:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760721.119537:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21745664, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa807 00010000:00010000:0.0:1789760721.119547:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545458f7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.119563:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760721.119567:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760721.119574:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545458f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.119589:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545458f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760721.215691:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21745664 -> 21749760 00010000:00010000:0.0:1789760721.215709:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545458f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.215722:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545458f7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.215729:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545458f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.215737:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760721.215742:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545458f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.215751:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545458f7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.215770:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545458f7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.215778:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545458f7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760721.215923:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495758336:12345-192.168.203.9@tcp:4:dd.0 Request processed in 96442us (97184us total) trans 4294972626 rc 0/0 00000100:00100000:0.0:1789760721.215939:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5338 00000100:00100000:0.0:1789760721.215949:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705364849763 time 705440180297 state 0x2 ref 0 00000001:00080000:0.0:1789760721.215987:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972626 is committed 00000100:00100000:2.0:1789760721.223548:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760721.223646:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495758464 00000100:00100000:1.0:1789760721.223656:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495758464 00000100:00100000:1.0:1789760721.223707:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705364849763 time 705440180297 state 0x0 ref 1 00000100:00100000:1.0:1789760721.223722:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705440440713 time 705440180297 state 0x1 ref 1 00000100:00100000:1.0:1789760721.223752:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705440440713 time 705544469313 state 0x0 ref 1 00000100:00100000:1.0:1789760721.223765:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495758464:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760721.223811:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21749760, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa807 00010000:00010000:1.0:1789760721.223820:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458fe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.223835:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760721.223840:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.223846:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.223860:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760721.294103:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972627 is committed 00002000:00010000:3.0:1789760721.294334:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21749760 -> 21753856 00010000:00010000:3.0:1789760721.294349:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.294362:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458fe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.294372:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.294380:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760721.294386:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.294396:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458fe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.294415:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458fe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.294440:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545458fe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760721.294528:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495758464:12345-192.168.203.9@tcp:4:dd.0 Request processed in 70761us (70985us total) trans 4294972627 rc 0/0 00000100:00100000:3.0:1789760721.294540:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5339 00000100:00100000:3.0:1789760721.294550:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705440440713 time 705544469313 state 0x2 ref 0 00000100:00100000:2.0:1789760721.305161:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760721.305332:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495758592 00000100:00100000:1.0:1789760721.305341:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495758592 00000100:00100000:1.0:1789760721.305399:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705440440713 time 705544469313 state 0x0 ref 1 00000100:00100000:1.0:1789760721.305413:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705544729729 time 705544469313 state 0x1 ref 1 00000100:00100000:1.0:1789760721.305453:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705544729729 time 705626169064 state 0x0 ref 1 00000100:00100000:1.0:1789760721.305466:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495758592:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760721.305515:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21753856, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa807 00010000:00010000:1.0:1789760721.305524:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545905 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.305539:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760721.305544:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.305550:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545905 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.305565:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545905 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760721.358638:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972628 is committed 00002000:00010000:3.0:1789760721.358775:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21753856 -> 21757952 00010000:00010000:3.0:1789760721.358792:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545905 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.358804:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545905 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.358814:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545905 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.358822:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760721.358827:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545905 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.358836:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545905 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.358859:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545905 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.358866:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545905 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760721.358992:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495758592:12345-192.168.203.9@tcp:4:dd.0 Request processed in 53523us (53834us total) trans 4294972628 rc 0/0 00000100:00100000:3.0:1789760721.359001:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5340 00000100:00100000:3.0:1789760721.359011:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705544729729 time 705626169064 state 0x2 ref 0 00000100:00100000:0.0:1789760721.366592:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760721.366737:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495758720 00000100:00100000:1.0:1789760721.366748:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495758720 00000100:00100000:1.0:1789760721.366803:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705544729729 time 705626169064 state 0x0 ref 1 00000100:00100000:1.0:1789760721.366819:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705626429480 time 705626169064 state 0x1 ref 1 00000100:00100000:1.0:1789760721.366851:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705626429480 time 705687567004 state 0x0 ref 1 00000100:00100000:1.0:1789760721.366867:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495758720:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760721.366914:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21757952, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa807 00010000:00010000:1.0:1789760721.366924:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454590c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.366941:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760721.366946:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.366952:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454590c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.366966:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454590c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760721.404959:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972629 is committed 00002000:00010000:2.0:1789760721.405432:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21757952 -> 21762048 00010000:00010000:2.0:1789760721.405447:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454590c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.405460:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454590c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.405470:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454590c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.405479:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760721.405484:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454590c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.405493:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454590c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.405512:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454590c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.405520:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454590c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760721.405617:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495758720:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38748us (39029us total) trans 4294972629 rc 0/0 00000100:00100000:2.0:1789760721.405627:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5341 00000100:00100000:2.0:1789760721.405637:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705626429480 time 705687567004 state 0x2 ref 0 00000100:00100000:2.0:1789760721.408873:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760721.409063:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495758848 00000100:00100000:3.0:1789760721.409091:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495758848 00000100:00100000:3.0:1789760721.409146:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705626429480 time 705687567004 state 0x0 ref 1 00000100:00100000:3.0:1789760721.409161:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705687827420 time 705687567004 state 0x1 ref 1 00000100:00100000:3.0:1789760721.409206:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705687827420 time 705729922282 state 0x0 ref 1 00000100:00100000:3.0:1789760721.409217:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495758848:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760721.409265:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21762048, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa807 00010000:00010000:3.0:1789760721.409274:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545913 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.409323:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760721.409328:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760721.409335:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545913 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.409349:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545913 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760721.458563:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972630 is committed 00002000:00010000:2.0:1789760721.458724:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21762048 -> 21766144 00010000:00010000:2.0:1789760721.458738:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545913 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.458750:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545913 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.458758:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545913 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.458765:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760721.458769:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545913 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.458776:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545913 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.458794:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545913 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.458806:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545913 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760721.458905:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495758848:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49685us (50042us total) trans 4294972630 rc 0/0 00000100:00100000:2.0:1789760721.458915:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5342 00000100:00100000:2.0:1789760721.458925:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705687827420 time 705729922282 state 0x2 ref 0 00000100:00100000:3.0:1789760721.466690:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760721.467488:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495758976 00000100:00100000:1.0:1789760721.467502:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495758976 00000100:00100000:1.0:1789760721.467576:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705687827420 time 705729922282 state 0x0 ref 1 00000100:00100000:1.0:1789760721.467593:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705730182698 time 705729922282 state 0x1 ref 1 00000100:00100000:1.0:1789760721.467678:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705730182698 time 705788394829 state 0x0 ref 1 00000100:00100000:1.0:1789760721.467692:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495758976:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760721.467738:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21766144, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa807 00010000:00010000:1.0:1789760721.467747:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454591a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.467765:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760721.467769:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.467776:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454591a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.467789:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454591a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760721.548997:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21766144 -> 21770240 00010000:00010000:1.0:1789760721.549019:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454591a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.549033:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454591a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.549042:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454591a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.549050:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760721.549055:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454591a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.549065:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454591a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.549084:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454591a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.549092:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454591a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.549185:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495758976:12345-192.168.203.9@tcp:4:dd.0 Request processed in 81491us (82501us total) trans 4294972631 rc 0/0 00000100:00100000:1.0:1789760721.549203:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5343 00000100:00100000:1.0:1789760721.549212:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705730182698 time 705788394829 state 0x2 ref 0 00000001:00080000:3.0:1789760721.550087:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972631 is committed 00000100:00100000:0.0:1789760721.552826:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760721.552948:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495759104 00000100:00100000:2.0:1789760721.552957:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495759104 00000100:00100000:2.0:1789760721.553016:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705730182698 time 705788394829 state 0x0 ref 1 00000100:00100000:2.0:1789760721.553033:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705788655245 time 705788394829 state 0x1 ref 1 00000100:00100000:2.0:1789760721.553093:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705788655245 time 705873808681 state 0x0 ref 1 00000100:00100000:2.0:1789760721.553106:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495759104:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760721.553153:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21770240, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa807 00010000:00010000:2.0:1789760721.553162:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545921 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.553199:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760721.553204:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760721.553211:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545921 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.553232:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545921 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760721.582468:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972632 is committed 00002000:00010000:2.0:1789760721.582640:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21770240 -> 21774336 00010000:00010000:2.0:1789760721.582656:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545921 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.582670:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545921 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.582681:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545921 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.582689:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760721.582695:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545921 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.582704:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545921 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.582723:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545921 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.582731:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545921 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760721.582864:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495759104:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29755us (30042us total) trans 4294972632 rc 0/0 00000100:00100000:2.0:1789760721.582876:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5344 00000100:00100000:2.0:1789760721.582886:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705788655245 time 705873808681 state 0x2 ref 0 00000100:00100000:2.0:1789760721.587008:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760721.587132:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495759232 00000100:00100000:3.0:1789760721.587141:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495759232 00000100:00100000:3.0:1789760721.587215:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705788655245 time 705873808681 state 0x0 ref 1 00000100:00100000:3.0:1789760721.587233:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705874069097 time 705873808681 state 0x1 ref 1 00000100:00100000:3.0:1789760721.587274:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705874069097 time 705907989007 state 0x0 ref 1 00000100:00100000:3.0:1789760721.587344:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495759232:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760721.587406:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21774336, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa807 00010000:00010000:3.0:1789760721.587417:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545928 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.587435:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760721.587440:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760721.587447:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545928 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.587468:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545928 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760721.618676:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972633 is committed 00002000:00010000:3.0:1789760721.618781:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21774336 -> 21778432 00010000:00010000:3.0:1789760721.618797:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545928 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.618811:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545928 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.618821:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545928 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.618828:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760721.618833:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545928 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.618842:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545928 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.618864:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545928 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.618872:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545928 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760721.618976:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495759232:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31632us (31973us total) trans 4294972633 rc 0/0 00000100:00100000:3.0:1789760721.618987:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5345 00000100:00100000:3.0:1789760721.618996:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705874069097 time 705907989007 state 0x2 ref 0 00000100:00100000:3.0:1789760721.622347:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760721.622473:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495759360 00000100:00100000:1.0:1789760721.622483:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495759360 00000100:00100000:1.0:1789760721.622603:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705874069097 time 705907989007 state 0x0 ref 1 00000100:00100000:1.0:1789760721.622619:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705908249423 time 705907989007 state 0x1 ref 1 00000100:00100000:1.0:1789760721.622644:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705908249423 time 705943360660 state 0x0 ref 1 00000100:00100000:1.0:1789760721.622657:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495759360:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760721.622706:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21778432, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa807 00010000:00010000:1.0:1789760721.622715:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454592f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.622730:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760721.622735:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.622741:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454592f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.622761:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454592f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760721.665233:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21778432 -> 21782528 00002000:00010000:1.0:1789760721.665243:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760720 -> 1789760721 00002000:00010000:1.0:1789760721.665246:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760720 -> 1789760721 00010000:00010000:1.0:1789760721.665260:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454592f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.665272:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454592f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.665312:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454592f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.665320:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760721.665325:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454592f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.665334:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454592f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.665354:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454592f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.665368:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454592f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.665459:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495759360:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42799us (43116us total) trans 4294972634 rc 0/0 00000100:00100000:1.0:1789760721.665469:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5346 00000100:00100000:1.0:1789760721.665479:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705908249423 time 705943360660 state 0x2 ref 0 00000100:00100000:1.0:1789760721.668939:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760721.669025:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495759488 00000100:00100000:3.0:1789760721.669036:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495759488 00000100:00100000:3.0:1789760721.669092:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705908249423 time 705943360660 state 0x0 ref 1 00000100:00100000:3.0:1789760721.669108:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705943621076 time 705943360660 state 0x1 ref 1 00000100:00100000:3.0:1789760721.669144:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705943621076 time 705989859971 state 0x0 ref 1 00000100:00100000:3.0:1789760721.669155:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495759488:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760721.669219:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21782528, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa807 00010000:00010000:3.0:1789760721.669228:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545936 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.669244:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760721.669248:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760721.669255:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545936 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.669269:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545936 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760721.669959:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972634 is committed 00002000:00010000:3.0:1789760721.697845:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21782528 -> 21786624 00010000:00010000:3.0:1789760721.697865:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545936 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.697879:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545936 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.697890:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545936 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.697897:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760721.697902:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545936 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.697913:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545936 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.697934:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545936 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.697941:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545936 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760721.698024:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495759488:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28865us (29090us total) trans 4294972635 rc 0/0 00000100:00100000:3.0:1789760721.698038:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5347 00000100:00100000:3.0:1789760721.698048:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705943621076 time 705989859971 state 0x2 ref 0 00000001:00080000:3.0:1789760721.698084:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972635 is committed 00000100:00100000:3.0:1789760721.701268:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760721.705907:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495759616 00000100:00100000:2.0:1789760721.705921:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495759616 00000100:00100000:2.0:1789760721.705987:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705943621076 time 705989859971 state 0x0 ref 1 00000100:00100000:2.0:1789760721.706004:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705990120387 time 705989859971 state 0x1 ref 1 00000100:00100000:2.0:1789760721.706040:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705990120387 time 706026756908 state 0x0 ref 1 00000100:00100000:2.0:1789760721.706055:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04931f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495759616:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760721.706101:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21786624, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa807 00010000:00010000:2.0:1789760721.706111:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454593d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.706128:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760721.706132:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760721.706140:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454593d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.706160:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454593d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760721.743436:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972636 is committed 00002000:00010000:3.0:1789760721.743605:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21786624 -> 21790720 00010000:00010000:3.0:1789760721.743622:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454593d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.743635:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454593d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.743644:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454593d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.743652:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760721.743658:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454593d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.743667:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454593d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.743688:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454593d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.743696:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454593d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760721.743859:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495759616:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37802us (42594us total) trans 4294972636 rc 0/0 00000100:00100000:3.0:1789760721.743870:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5348 00000100:00100000:3.0:1789760721.743880:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705990120387 time 706026756908 state 0x2 ref 0 00000100:00100000:3.0:1789760721.750181:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760721.750333:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495759744 00000100:00100000:1.0:1789760721.750343:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495759744 00000100:00100000:1.0:1789760721.750402:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 705990120387 time 706026756908 state 0x0 ref 1 00000100:00100000:1.0:1789760721.750418:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706027017324 time 706026756908 state 0x1 ref 1 00000100:00100000:1.0:1789760721.750441:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706027017324 time 706071158284 state 0x0 ref 1 00000100:00100000:1.0:1789760721.750454:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04933480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495759744:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760721.750501:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21790720, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa807 00010000:00010000:1.0:1789760721.750510:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545944 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.750527:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760721.750532:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.750539:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545944 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.750554:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545944 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.760690:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760721.826891:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972637 is committed 00002000:00010000:1.0:1789760721.827059:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21790720 -> 21794816 00010000:00010000:1.0:1789760721.827074:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545944 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.827087:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545944 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.827096:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545944 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.827103:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760721.827108:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545944 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.827117:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545944 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.827137:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545944 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.827145:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545944 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.827347:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04933480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495759744:12345-192.168.203.9@tcp:4:dd.0 Request processed in 76890us (77181us total) trans 4294972637 rc 0/0 00000100:00100000:1.0:1789760721.827357:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5349 00000100:00100000:1.0:1789760721.827367:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706027017324 time 706071158284 state 0x2 ref 0 00000100:00100000:1.0:1789760721.831927:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760721.832677:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495759872 00000100:00100000:3.0:1789760721.832689:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495759872 00000100:00100000:3.0:1789760721.832746:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706027017324 time 706071158284 state 0x0 ref 1 00000100:00100000:3.0:1789760721.832772:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706071418700 time 706071158284 state 0x1 ref 1 00000100:00100000:3.0:1789760721.832804:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706071418700 time 706153521253 state 0x0 ref 1 00000100:00100000:3.0:1789760721.832817:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495759872:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760721.832865:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21794816, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa807 00010000:00010000:3.0:1789760721.832874:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454594b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.832890:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760721.832894:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760721.832902:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454594b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.832919:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454594b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760721.869426:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972638 is committed 00002000:00010000:3.0:1789760721.871031:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21794816 -> 21798912 00010000:00010000:3.0:1789760721.871050:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454594b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.871063:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454594b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.871071:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454594b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.871078:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760721.871083:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454594b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.871092:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454594b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.871112:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454594b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.871119:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454594b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760721.871816:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495759872:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38993us (39899us total) trans 4294972638 rc 0/0 00000100:00100000:2.0:1789760721.871828:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5350 00000100:00100000:2.0:1789760721.871839:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706071418700 time 706153521253 state 0x2 ref 0 00000100:00100000:0.0:1789760721.877863:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760721.877958:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495760000 00000100:00100000:3.0:1789760721.877968:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495760000 00000100:00100000:3.0:1789760721.878022:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706071418700 time 706153521253 state 0x0 ref 1 00000100:00100000:3.0:1789760721.878038:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706153781669 time 706153521253 state 0x1 ref 1 00000100:00100000:3.0:1789760721.878067:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706153781669 time 706198784064 state 0x0 ref 1 00000100:00100000:3.0:1789760721.878081:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495760000:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760721.878129:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21798912, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa807 00010000:00010000:3.0:1789760721.878139:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545952 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.878155:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760721.878160:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760721.878166:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545952 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.878180:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545952 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760721.922366:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972639 is committed 00002000:00010000:3.0:1789760721.922728:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21798912 -> 21803008 00010000:00010000:3.0:1789760721.922744:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545952 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.922763:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545952 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.922771:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545952 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.922779:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760721.922784:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545952 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.922793:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545952 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.922812:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545952 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.922820:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545952 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760721.922913:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495760000:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44831us (45054us total) trans 4294972639 rc 0/0 00000100:00100000:3.0:1789760721.922923:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5351 00000100:00100000:3.0:1789760721.922933:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706153781669 time 706198784064 state 0x2 ref 0 00000100:00100000:0.0:1789760721.926383:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760721.926579:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495760128 00000100:00100000:1.0:1789760721.926589:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495760128 00000100:00100000:1.0:1789760721.926777:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706153781669 time 706198784064 state 0x0 ref 1 00000100:00100000:1.0:1789760721.926795:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706199044480 time 706198784064 state 0x1 ref 1 00000100:00100000:1.0:1789760721.926832:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706199044480 time 706247548445 state 0x0 ref 1 00000100:00100000:1.0:1789760721.926847:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495760128:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760721.926901:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21803008, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa807 00010000:00010000:1.0:1789760721.926912:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545959 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.926930:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760721.926934:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.926941:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545959 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.926956:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545959 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760721.962713:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972640 is committed 00002000:00010000:0.0:1789760721.962874:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21803008 -> 21807104 00010000:00010000:0.0:1789760721.962889:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545959 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.962900:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545959 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.962909:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545959 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.962916:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760721.962922:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545959 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.962935:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545959 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.962956:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545959 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.962963:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545959 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760721.963059:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495760128:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36208us (36680us total) trans 4294972640 rc 0/0 00000100:00100000:0.0:1789760721.963070:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5352 00000100:00100000:0.0:1789760721.963080:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706199044480 time 706247548445 state 0x2 ref 0 00000100:00100000:0.0:1789760721.968593:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760721.968776:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495760256 00000100:00100000:2.0:1789760721.968787:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495760256 00000100:00100000:2.0:1789760721.968848:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706199044480 time 706247548445 state 0x0 ref 1 00000100:00100000:2.0:1789760721.968864:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706247808861 time 706247548445 state 0x1 ref 1 00000100:00100000:2.0:1789760721.968927:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706247808861 time 706289642535 state 0x0 ref 1 00000100:00100000:2.0:1789760721.968939:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495760256:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760721.968988:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21807104, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa807 00010000:00010000:2.0:1789760721.969000:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545960 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.969016:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760721.969021:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760721.969027:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545960 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.969047:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545960 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760721.998420:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972641 is committed 00002000:00010000:0.0:1789760721.998590:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21807104 -> 21811200 00010000:00010000:0.0:1789760721.998605:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545960 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.998619:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545960 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.998628:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545960 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.998702:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760721.998709:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545960 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.998720:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545960 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.998740:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545960 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.998748:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545960 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760721.998841:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495760256:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29899us (30254us total) trans 4294972641 rc 0/0 00000100:00100000:0.0:1789760721.998852:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5353 00000100:00100000:0.0:1789760721.998863:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706247808861 time 706289642535 state 0x2 ref 0 00000100:00100000:3.0:1789760722.002636:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760722.002896:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495760384 00000100:00100000:3.0:1789760722.002909:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495760384 00000100:00100000:3.0:1789760722.002976:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706247808861 time 706289642535 state 0x0 ref 1 00000100:00100000:3.0:1789760722.002993:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706289902951 time 706289642535 state 0x1 ref 1 00000100:00100000:3.0:1789760722.003021:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706289902951 time 706323737907 state 0x0 ref 1 00000100:00100000:3.0:1789760722.003034:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04932680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495760384:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760722.003082:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21811200, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa807 00010000:00010000:3.0:1789760722.003092:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545967 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.003107:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760722.003112:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760722.003118:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545967 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.003132:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545967 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760722.044522:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972642 is committed 00002000:00010000:0.0:1789760722.044721:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21811200 -> 21815296 00010000:00010000:0.0:1789760722.044737:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545967 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.044750:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545967 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.044765:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545967 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.044772:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760722.044777:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545967 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.044787:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545967 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.045012:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545967 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.045022:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545967 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760722.045161:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04932680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495760384:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42124us (42532us total) trans 4294972642 rc 0/0 00000100:00100000:0.0:1789760722.045171:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5354 00000100:00100000:0.0:1789760722.045181:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706289902951 time 706323737907 state 0x2 ref 0 00000100:00100000:3.0:1789760722.050096:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760722.050712:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495760512 00000100:00100000:0.0:1789760722.050722:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495760512 00000100:00100000:0.0:1789760722.050777:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706289902951 time 706323737907 state 0x0 ref 1 00000100:00100000:0.0:1789760722.050793:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706323998323 time 706323737907 state 0x1 ref 1 00000100:00100000:0.0:1789760722.050884:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706323998323 time 706371599984 state 0x0 ref 1 00000100:00100000:0.0:1789760722.050898:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04930a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495760512:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760722.050948:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21815296, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa807 00010000:00010000:0.0:1789760722.050957:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454596e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.050973:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760722.050978:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.050985:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454596e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.050999:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454596e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760722.076703:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21815296 -> 21819392 00010000:00010000:1.0:1789760722.076727:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454596e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.076742:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454596e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.076751:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454596e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.076759:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760722.076768:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454596e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.076779:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454596e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.076799:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454596e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.076807:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454596e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.076903:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04930a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495760512:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26004us (26811us total) trans 4294972643 rc 0/0 00000100:00100000:1.0:1789760722.076915:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5355 00000100:00100000:1.0:1789760722.076924:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706323998323 time 706371599984 state 0x2 ref 0 00000001:00080000:1.0:1789760722.076965:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972643 is committed 00000100:00100000:1.0:1789760722.088470:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760722.088551:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495760640 00000100:00100000:0.0:1789760722.088561:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495760640 00000100:00100000:0.0:1789760722.088613:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706323998323 time 706371599984 state 0x0 ref 1 00000100:00100000:0.0:1789760722.088630:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706371860400 time 706371599984 state 0x1 ref 1 00000100:00100000:0.0:1789760722.088660:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706371860400 time 706409376735 state 0x0 ref 1 00000100:00100000:0.0:1789760722.088673:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495760640:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760722.088720:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21819392, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa807 00010000:00010000:0.0:1789760722.088729:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545975 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.088745:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760722.088750:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.088756:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545975 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.088770:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545975 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760722.122867:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972644 is committed 00002000:00010000:1.0:1789760722.122934:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21819392 -> 21823488 00010000:00010000:1.0:1789760722.122954:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545975 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.122966:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545975 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.122974:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545975 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.122983:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760722.122988:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545975 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.122997:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545975 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.123015:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545975 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.123023:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545975 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.123107:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495760640:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34432us (34641us total) trans 4294972644 rc 0/0 00000100:00100000:1.0:1789760722.123116:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5356 00000100:00100000:1.0:1789760722.123124:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706371860400 time 706409376735 state 0x2 ref 0 00000100:00100000:2.0:1789760722.131056:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760722.131135:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495760768 00000100:00100000:0.0:1789760722.131147:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495760768 00000100:00100000:0.0:1789760722.131219:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706371860400 time 706409376735 state 0x0 ref 1 00000100:00100000:0.0:1789760722.131236:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706409637151 time 706409376735 state 0x1 ref 1 00000100:00100000:0.0:1789760722.131395:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706409637151 time 706452110537 state 0x0 ref 1 00000100:00100000:0.0:1789760722.131412:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495760768:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760722.131587:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21823488, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa807 00010000:00010000:0.0:1789760722.131598:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454597c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.131614:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760722.131620:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.131627:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454597c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.131641:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454597c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760722.191090:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972645 is committed 00002000:00010000:2.0:1789760722.191334:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21823488 -> 21827584 00010000:00010000:2.0:1789760722.191361:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454597c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.191375:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454597c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.191385:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454597c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.191392:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ae00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760722.191396:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454597c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.191405:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454597c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.191425:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454597c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.191433:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454597c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.191524:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495760768:12345-192.168.203.9@tcp:4:dd.0 Request processed in 60111us (60477us total) trans 4294972645 rc 0/0 00000100:00100000:2.0:1789760722.191534:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5357 00000100:00100000:2.0:1789760722.191544:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706409637151 time 706452110537 state 0x2 ref 0 00000100:00100000:3.0:1789760722.196516:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760722.196627:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495760896 00000100:00100000:1.0:1789760722.196637:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495760896 00000100:00100000:1.0:1789760722.196696:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706409637151 time 706452110537 state 0x0 ref 1 00000100:00100000:1.0:1789760722.196712:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706452370953 time 706452110537 state 0x1 ref 1 00000100:00100000:1.0:1789760722.196844:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706452370953 time 706517559530 state 0x0 ref 1 00000100:00100000:1.0:1789760722.196856:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04931180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495760896:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760722.196902:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21827584, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa807 00010000:00010000:1.0:1789760722.196911:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354545983 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.196926:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760722.196929:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.196936:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354545983 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.196948:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354545983 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.206493:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033cdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520303232:0@lo:400:kworker.0 00000100:00100000:2.0:1789760722.206562:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760722.206632:0:8003:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520303232 00000100:00100000:2.0:1789760722.206640:0:8003:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520303232 00000100:00100000:2.0:1789760722.206686:0:8003:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 462 00000100:00100000:2.0:1789760722.206693:0:8003:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520303232:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760722.206738:0:8003:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520303232:12345-0@lo:400:kworker.0 Request processed in 46us (176us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760722.206745:0:8003:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 462 00000100:00080000:2.0:1789760722.206763:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760722.206789:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033cdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520303232:0@lo:400:kworker.0 00000100:00100000:0.0:1789760722.206911:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:59c4e264-9b75-482b-8813-3d1dc124f384:3317:1876699520302976:0@lo:400:kworker.0 00000100:00100000:0.0:1789760722.206954:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760722.206993:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760722.207000:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033cc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3317:1876699520303104:0@lo:400:kworker.0 00000100:00100000:0.0:1789760722.207032:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760722.207551:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520303104 00000100:00100000:1.0:1789760722.207557:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520303104 00000100:00100000:1.0:1789760722.207583:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 463 00000100:00100000:1.0:1789760722.207589:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92902dbc4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1876699520303104:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760722.207633:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92902dbc4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1876699520303104:12345-0@lo:400:kworker.0 Request processed in 43us (600us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760722.207640:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 463 00000100:00100000:2.0:1789760722.207956:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520302976 00000100:00100000:2.0:1789760722.207961:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520302976 00000100:00100000:2.0:1789760722.207990:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 393 00000100:00100000:2.0:1789760722.207996:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929012315050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:59c4e264-9b75-482b-8813-3d1dc124f384+19:3317:x1876699520302976:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760722.208028:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929012315050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:59c4e264-9b75-482b-8813-3d1dc124f384+19:3317:x1876699520302976:12345-0@lo:400:kworker.0 Request processed in 32us (1075us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760722.208034:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 393 00000100:00100000:2.0:1789760722.208064:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789760722.208073:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033ce300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520303360:0@lo:400:kworker.0 00000100:00100000:1.0:1789760722.208088:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3318:1876699520303616:0@lo:400:kworker.0 00000100:00100000:2.0:1789760722.208101:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760722.208114:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033cea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3317:1876699520303488:0@lo:400:kworker.0 00000100:00100000:2.0:1789760722.208130:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520303360 00000100:00100000:2.0:1789760722.208133:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520303360 00000100:00100000:1.0:1789760722.208136:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760722.208163:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 216 00000100:00100000:1.0:1789760722.208164:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033cc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3317:1876699520303104:0@lo:400:kworker.0 00000100:00100000:2.0:1789760722.208169:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520303360:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760722.208180:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:59c4e264-9b75-482b-8813-3d1dc124f384:3317:1876699520302976:0@lo:400:kworker.0 00000100:00100000:2.0:1789760722.208213:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520303360:12345-0@lo:400:kworker.0 Request processed in 44us (110us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760722.208214:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520303488 00000100:00100000:1.0:1789760722.208217:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520303488 00000100:00100000:2.0:1789760722.208219:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 216 00000100:00080000:2.0:1789760722.208232:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789760722.208235:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760722.208240:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 464 00000100:00100000:2.0:1789760722.208248:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760722.208272:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520303616 00000100:00100000:2.0:1789760722.208276:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520303616 00000100:00100000:2.0:1789760722.208319:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 217 00000100:00100000:2.0:1789760722.208325:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3318:x1876699520303616:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760722.208446:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1876699520303488:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760722.208486:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1876699520303488:12345-0@lo:400:kworker.0 Request processed in 240us (348us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760722.208492:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 464 00000100:00100000:2.0:1789760722.208492:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3318:x1876699520303616:12345-0@lo:400:kworker.0 Request processed in 168us (243us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760722.208499:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 217 00000100:00080000:2.0:1789760722.208512:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789760722.208516:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033cea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3317:1876699520303488:0@lo:400:kworker.0 00000100:00100000:2.0:1789760722.208525:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3318:1876699520303616:0@lo:400:kworker.0 00000100:00100000:2.0:1789760722.208543:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033ce300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520303360:0@lo:400:kworker.0 00000001:00080000:0.0:1789760722.250067:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972646 is committed 00002000:00010000:2.0:1789760722.250416:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21827584 -> 21831680 00010000:00010000:2.0:1789760722.250432:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354545983 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.250444:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354545983 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.250453:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354545983 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.250461:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760722.250466:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354545983 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.250476:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354545983 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.250496:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354545983 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.250505:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354545983 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.250652:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495760896:12345-192.168.203.9@tcp:4:dd.0 Request processed in 53792us (54140us total) trans 4294972646 rc 0/0 00000100:00100000:2.0:1789760722.250663:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5358 00000100:00100000:2.0:1789760722.250675:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706452370953 time 706517559530 state 0x2 ref 0 00000100:00100000:1.0:1789760722.265424:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760722.265506:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495761024 00000100:00100000:0.0:1789760722.265516:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495761024 00000100:00100000:0.0:1789760722.265569:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706452370953 time 706517559530 state 0x0 ref 1 00000100:00100000:0.0:1789760722.265584:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706517819946 time 706517559530 state 0x1 ref 1 00000100:00100000:0.0:1789760722.265671:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706517819946 time 706586387358 state 0x0 ref 1 00000100:00100000:0.0:1789760722.265685:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495761024:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760722.265731:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21831680, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa807 00010000:00010000:0.0:1789760722.265740:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454598a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.265756:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760722.265761:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.265768:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454598a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.265781:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454598a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760722.300919:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972647 is committed 00002000:00010000:2.0:1789760722.302548:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21831680 -> 21835776 00010000:00010000:2.0:1789760722.302567:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454598a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.302585:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454598a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.302593:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454598a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.302599:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760722.302604:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454598a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.302612:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454598a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.302631:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454598a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.302637:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454598a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.302792:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495761024:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37105us (37373us total) trans 4294972647 rc 0/0 00000100:00100000:2.0:1789760722.302802:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5359 00000100:00100000:2.0:1789760722.302811:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706517819946 time 706586387358 state 0x2 ref 0 00000100:00100000:0.0:1789760722.317666:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760722.317774:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495761152 00000100:00100000:2.0:1789760722.317784:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495761152 00000100:00100000:2.0:1789760722.317841:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706517819946 time 706586387358 state 0x0 ref 1 00000100:00100000:2.0:1789760722.317856:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706586647774 time 706586387358 state 0x1 ref 1 00000100:00100000:2.0:1789760722.317892:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706586647774 time 706638607787 state 0x0 ref 1 00000100:00100000:2.0:1789760722.317905:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495761152:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760722.317955:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21835776, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa807 00010000:00010000:2.0:1789760722.317964:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545991 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.317979:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760722.317983:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.317989:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545991 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.318001:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545991 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760722.348083:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972648 is committed 00002000:00010000:2.0:1789760722.348307:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21835776 -> 21839872 00010000:00010000:2.0:1789760722.348323:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545991 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.348336:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545991 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.348352:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545991 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.348360:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760722.348366:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545991 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.348375:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545991 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.348393:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545991 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.348400:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545991 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.348575:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495761152:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30667us (30912us total) trans 4294972648 rc 0/0 00000100:00100000:2.0:1789760722.348585:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5360 00000100:00100000:2.0:1789760722.348594:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706586647774 time 706638607787 state 0x2 ref 0 00000100:00100000:3.0:1789760722.351651:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760722.351765:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495761280 00000100:00100000:2.0:1789760722.351775:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495761280 00000100:00100000:2.0:1789760722.351826:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706586647774 time 706638607787 state 0x0 ref 1 00000100:00100000:2.0:1789760722.351841:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706638868203 time 706638607787 state 0x1 ref 1 00000100:00100000:2.0:1789760722.351931:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706638868203 time 706672646419 state 0x0 ref 1 00000100:00100000:2.0:1789760722.351944:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04932300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495761280:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760722.351995:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21839872, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa807 00010000:00010000:2.0:1789760722.352004:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545998 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.352019:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760722.352024:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.352031:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545998 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.352044:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545998 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760722.437234:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972649 is committed 00002000:00010000:2.0:1789760722.437423:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21839872 -> 21843968 00010000:00010000:2.0:1789760722.437439:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545998 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.437452:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545998 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.437461:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545998 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.437469:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760722.437474:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545998 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.437484:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545998 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.437506:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545998 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.437514:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545998 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.437622:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04932300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495761280:12345-192.168.203.9@tcp:4:dd.0 Request processed in 85674us (85974us total) trans 4294972649 rc 0/0 00000100:00100000:2.0:1789760722.437633:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5361 00000100:00100000:2.0:1789760722.437655:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706638868203 time 706672646419 state 0x2 ref 0 00000100:00100000:1.0:1789760722.443061:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760722.443135:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495761408 00000100:00100000:2.0:1789760722.443144:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495761408 00000100:00100000:2.0:1789760722.443207:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706638868203 time 706672646419 state 0x0 ref 1 00000100:00100000:2.0:1789760722.443222:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706672906835 time 706672646419 state 0x1 ref 1 00000100:00100000:2.0:1789760722.443250:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706672906835 time 706763967293 state 0x0 ref 1 00000100:00100000:2.0:1789760722.443263:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495761408:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760722.443345:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21843968, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa807 00010000:00010000:2.0:1789760722.443356:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454599f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.443372:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760722.443376:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.443383:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454599f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.443397:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454599f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760722.561874:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21843968 -> 21848064 00010000:00010000:2.0:1789760722.561898:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454599f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.561911:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454599f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.561920:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454599f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.561926:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760722.561942:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454599f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.561951:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454599f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.561975:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454599f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.561982:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454599f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.562082:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495761408:12345-192.168.203.9@tcp:4:dd.0 Request processed in 118816us (119026us total) trans 4294972650 rc 0/0 00000100:00100000:2.0:1789760722.562094:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5362 00000100:00100000:2.0:1789760722.562104:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706672906835 time 706763967293 state 0x2 ref 0 00000001:00080000:3.0:1789760722.562446:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972650 is committed 00000100:00100000:0.0:1789760722.568779:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760722.568850:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495761536 00000100:00100000:2.0:1789760722.568860:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495761536 00000100:00100000:2.0:1789760722.568910:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706672906835 time 706763967293 state 0x0 ref 1 00000100:00100000:2.0:1789760722.568925:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706764227709 time 706763967293 state 0x1 ref 1 00000100:00100000:2.0:1789760722.568959:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706764227709 time 706889675613 state 0x0 ref 1 00000100:00100000:2.0:1789760722.568971:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495761536:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760722.569017:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21848064, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa807 00010000:00010000:2.0:1789760722.569027:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545459a6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.569062:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760722.569067:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.569074:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545459a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.569087:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545459a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760722.663502:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972651 is committed 00002000:00010000:2.0:1789760722.664348:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21848064 -> 21852160 00010000:00010000:2.0:1789760722.664365:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545459a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.664377:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545459a6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.664385:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545459a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.664393:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760722.664398:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545459a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.664405:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545459a6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.664423:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545459a6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.664435:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545459a6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.664587:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495761536:12345-192.168.203.9@tcp:4:dd.0 Request processed in 95613us (95812us total) trans 4294972651 rc 0/0 00000100:00100000:2.0:1789760722.664599:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5363 00000100:00100000:2.0:1789760722.664608:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706764227709 time 706889675613 state 0x2 ref 0 00000100:00100000:1.0:1789760722.675536:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760722.675706:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495761664 00000100:00100000:2.0:1789760722.675717:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495761664 00000100:00100000:2.0:1789760722.675783:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706764227709 time 706889675613 state 0x0 ref 1 00000100:00100000:2.0:1789760722.675800:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706889936029 time 706889675613 state 0x1 ref 1 00000100:00100000:2.0:1789760722.675833:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706889936029 time 706996550097 state 0x0 ref 1 00000100:00100000:2.0:1789760722.675847:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495761664:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760722.675909:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21852160, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa807 00010000:00010000:2.0:1789760722.675920:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545459ad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.675939:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760722.675945:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.675952:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545459ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.675973:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545459ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760722.709044:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972652 is committed 00002000:00010000:3.0:1789760722.709373:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21852160 -> 21856256 00002000:00010000:3.0:1789760722.709378:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760721 -> 1789760722 00002000:00010000:3.0:1789760722.709381:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760721 -> 1789760722 00010000:00010000:3.0:1789760722.709394:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545459ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.709406:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545459ad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.709414:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545459ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.709422:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760722.709427:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545459ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.709436:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545459ad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.709457:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545459ad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.709465:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545459ad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760722.709559:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495761664:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33710us (34027us total) trans 4294972652 rc 0/0 00000100:00100000:3.0:1789760722.709569:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5364 00000100:00100000:3.0:1789760722.709580:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706889936029 time 706996550097 state 0x2 ref 0 00000100:00100000:1.0:1789760722.713267:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760722.713381:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495761792 00000100:00100000:0.0:1789760722.713391:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495761792 00000100:00100000:0.0:1789760722.713444:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706889936029 time 706996550097 state 0x0 ref 1 00000100:00100000:0.0:1789760722.713459:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706996810513 time 706996550097 state 0x1 ref 1 00000100:00100000:0.0:1789760722.713491:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706996810513 time 707034208216 state 0x0 ref 1 00000100:00100000:0.0:1789760722.713619:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495761792:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760722.713666:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21856256, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa807 00010000:00010000:0.0:1789760722.713676:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545459b4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.713693:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760722.713698:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.713704:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545459b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.713718:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545459b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760722.724445:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f04933b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520303744:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760722.724527:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760722.724599:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520303744 00000100:00100000:3.0:1789760722.724607:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520303744 00000100:00100000:3.0:1789760722.724647:0:6711:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 192 00000100:00100000:3.0:1789760722.724656:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04933100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520303744:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760722.724690:0:6711:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760722.724729:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04933100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520303744:12345-0@lo:13:osp-pre-1-0.0 Request processed in 73us (201us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760722.724736:0:6711:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 192 00000100:00100000:3.0:1789760722.724789:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f04933b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520303744:0@lo:13:osp-pre-1-0.0 00002000:00010000:1.0:1789760722.741375:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21856256 -> 21860352 00010000:00010000:1.0:1789760722.741395:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545459b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.741409:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545459b4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.741418:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545459b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.741427:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760722.741431:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545459b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.741440:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545459b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.741460:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545459b4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.741474:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545459b4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760722.741509:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972653 is committed 00000100:00100000:1.0:1789760722.741560:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495761792:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28054us (28298us total) trans 4294972653 rc 0/0 00000100:00100000:1.0:1789760722.741571:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5365 00000100:00100000:1.0:1789760722.741581:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706996810513 time 707034208216 state 0x2 ref 0 00000100:00100000:1.0:1789760722.746458:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760722.746495:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495761920 00000100:00100000:2.0:1789760722.746506:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495761920 00000100:00100000:2.0:1789760722.746561:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 706996810513 time 707034208216 state 0x0 ref 1 00000100:00100000:2.0:1789760722.746576:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707034468632 time 707034208216 state 0x1 ref 1 00000100:00100000:2.0:1789760722.746605:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707034468632 time 707067322159 state 0x0 ref 1 00000100:00100000:2.0:1789760722.746617:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495761920:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760722.746671:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21860352, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa807 00010000:00010000:2.0:1789760722.746680:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545459bb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.746696:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760722.746700:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.746707:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545459bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.746720:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545459bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760722.774689:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972654 is committed 00002000:00010000:3.0:1789760722.774873:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21860352 -> 21864448 00010000:00010000:3.0:1789760722.774890:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545459bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.774904:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545459bb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.774914:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545459bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.774921:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760722.774926:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545459bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.774936:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545459bb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.774956:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545459bb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.774965:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545459bb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760722.775058:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495761920:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28439us (28606us total) trans 4294972654 rc 0/0 00000100:00100000:3.0:1789760722.775069:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5366 00000100:00100000:3.0:1789760722.775079:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707034468632 time 707067322159 state 0x2 ref 0 00000100:00100000:3.0:1789760722.780171:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760722.780321:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495762048 00000100:00100000:2.0:1789760722.780331:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495762048 00000100:00100000:2.0:1789760722.780387:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707034468632 time 707067322159 state 0x0 ref 1 00000100:00100000:2.0:1789760722.780400:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707067582575 time 707067322159 state 0x1 ref 1 00000100:00100000:2.0:1789760722.780432:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707067582575 time 707101149163 state 0x0 ref 1 00000100:00100000:2.0:1789760722.780448:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04933b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495762048:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760722.780494:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21864448, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa807 00010000:00010000:2.0:1789760722.780504:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545459c2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.780521:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760722.780526:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.780532:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545459c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.780545:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545459c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.784772:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760722.816070:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972655 is committed 00002000:00010000:3.0:1789760722.818324:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21864448 -> 21868544 00010000:00010000:3.0:1789760722.818342:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545459c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.818354:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545459c2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.818364:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545459c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.818376:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760722.818381:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545459c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.818390:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545459c2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.818410:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545459c2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.818418:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545459c2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760722.818552:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04933b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495762048:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38101us (38391us total) trans 4294972655 rc 0/0 00000100:00100000:3.0:1789760722.818562:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5367 00000100:00100000:3.0:1789760722.818572:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707067582575 time 707101149163 state 0x2 ref 0 00000100:00100000:1.0:1789760722.821554:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760722.821655:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495762176 00000100:00100000:0.0:1789760722.821666:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495762176 00000100:00100000:0.0:1789760722.821718:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707067582575 time 707101149163 state 0x0 ref 1 00000100:00100000:0.0:1789760722.821733:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707101409579 time 707101149163 state 0x1 ref 1 00000100:00100000:0.0:1789760722.821763:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707101409579 time 707142479807 state 0x0 ref 1 00000100:00100000:0.0:1789760722.821775:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495762176:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760722.821821:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21868544, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa807 00010000:00010000:0.0:1789760722.821829:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545459c9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.821847:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760722.821852:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.821858:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545459c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.821872:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545459c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760722.846977:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972656 is committed 00002000:00010000:2.0:1789760722.847078:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21868544 -> 21872640 00010000:00010000:2.0:1789760722.847094:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545459c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.847106:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545459c9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.847119:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545459c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.847126:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760722.847131:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545459c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.847140:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545459c9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.847169:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545459c9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.847178:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545459c9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.847276:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495762176:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25497us (25725us total) trans 4294972656 rc 0/0 00000100:00100000:2.0:1789760722.847314:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5368 00000100:00100000:2.0:1789760722.847324:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707101409579 time 707142479807 state 0x2 ref 0 00000100:00100000:1.0:1789760722.852128:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760722.852321:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495762304 00000100:00100000:3.0:1789760722.852331:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495762304 00000100:00100000:3.0:1789760722.852384:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707101409579 time 707142479807 state 0x0 ref 1 00000100:00100000:3.0:1789760722.852398:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707142740223 time 707142479807 state 0x1 ref 1 00000100:00100000:3.0:1789760722.852426:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707142740223 time 707173142311 state 0x0 ref 1 00000100:00100000:3.0:1789760722.852437:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495762304:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760722.852484:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21872640, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa807 00010000:00010000:3.0:1789760722.852492:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545459d0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.852629:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760722.852634:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760722.852641:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545459d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.852656:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545459d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760722.878914:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972657 is committed 00002000:00010000:1.0:1789760722.879027:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21872640 -> 21876736 00010000:00010000:1.0:1789760722.879041:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545459d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.879054:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545459d0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.879063:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545459d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.879071:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760722.879075:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545459d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.879085:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545459d0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.879106:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545459d0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.879114:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545459d0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.879227:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495762304:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26786us (27105us total) trans 4294972657 rc 0/0 00000100:00100000:1.0:1789760722.879237:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5369 00000100:00100000:1.0:1789760722.879247:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707142740223 time 707173142311 state 0x2 ref 0 00000100:00100000:1.0:1789760722.884413:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760722.884486:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495762432 00000100:00100000:3.0:1789760722.884497:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495762432 00000100:00100000:3.0:1789760722.884561:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707142740223 time 707173142311 state 0x0 ref 1 00000100:00100000:3.0:1789760722.884577:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707173402727 time 707173142311 state 0x1 ref 1 00000100:00100000:3.0:1789760722.884608:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707173402727 time 707205325439 state 0x0 ref 1 00000100:00100000:3.0:1789760722.884620:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495762432:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760722.884665:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21876736, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa807 00010000:00010000:3.0:1789760722.884674:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545459d7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.884695:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760722.884699:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760722.884705:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545459d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.884718:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545459d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760722.919945:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972658 is committed 00002000:00010000:3.0:1789760722.920058:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21876736 -> 21880832 00010000:00010000:3.0:1789760722.920076:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545459d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.920090:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545459d7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.920099:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545459d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.920106:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760722.920111:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545459d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.920120:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545459d7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.920139:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545459d7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.920147:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545459d7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760722.920254:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495762432:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35631us (35845us total) trans 4294972658 rc 0/0 00000100:00100000:3.0:1789760722.920263:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5370 00000100:00100000:3.0:1789760722.920273:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707173402727 time 707205325439 state 0x2 ref 0 00000100:00100000:2.0:1789760722.925011:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760722.925174:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495762560 00000100:00100000:0.0:1789760722.925192:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495762560 00000100:00100000:0.0:1789760722.925247:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707173402727 time 707205325439 state 0x0 ref 1 00000100:00100000:0.0:1789760722.925261:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707205585855 time 707205325439 state 0x1 ref 1 00000100:00100000:0.0:1789760722.925319:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707205585855 time 707246034978 state 0x0 ref 1 00000100:00100000:0.0:1789760722.925331:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495762560:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760722.925377:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21880832, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa807 00010000:00010000:0.0:1789760722.925385:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545459de lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.925401:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760722.925405:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.925411:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545459de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.925424:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545459de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760722.956880:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21880832 -> 21884928 00010000:00010000:0.0:1789760722.956900:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545459de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.956915:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545459de lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.956924:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545459de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.956931:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760722.956936:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545459de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.956945:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545459de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.956964:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545459de lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.956972:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545459de lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760722.957064:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495762560:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31731us (32058us total) trans 4294972659 rc 0/0 00000100:00100000:0.0:1789760722.957075:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5371 00000100:00100000:0.0:1789760722.957085:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707205585855 time 707246034978 state 0x2 ref 0 00000001:00080000:0.0:1789760722.958474:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972659 is committed 00000100:00100000:1.0:1789760722.961662:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760722.961741:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495762688 00000100:00100000:3.0:1789760722.961751:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495762688 00000100:00100000:3.0:1789760722.961802:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707205585855 time 707246034978 state 0x0 ref 1 00000100:00100000:3.0:1789760722.961817:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707246295394 time 707246034978 state 0x1 ref 1 00000100:00100000:3.0:1789760722.961843:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707246295394 time 707282559998 state 0x0 ref 1 00000100:00100000:3.0:1789760722.961856:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495762688:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760722.961901:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21884928, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa807 00010000:00010000:3.0:1789760722.961909:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545459e5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.961925:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760722.961928:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760722.961934:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545459e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.961950:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545459e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760722.991140:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21884928 -> 21889024 00010000:00010000:0.0:1789760722.991226:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545459e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.991239:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545459e5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.991247:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545459e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.991258:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760722.991263:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545459e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.991272:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545459e5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.991309:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545459e5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.991316:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545459e5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760722.991406:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495762688:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29548us (29749us total) trans 4294972660 rc 0/0 00000100:00100000:0.0:1789760722.991415:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5372 00000100:00100000:0.0:1789760722.991424:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707246295394 time 707282559998 state 0x2 ref 0 00000001:00080000:0.0:1789760722.991459:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972660 is committed 00000100:00100000:3.0:1789760722.999610:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760722.999695:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495762816 00000100:00100000:0.0:1789760722.999706:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495762816 00000100:00100000:0.0:1789760722.999763:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707246295394 time 707282559998 state 0x0 ref 1 00000100:00100000:0.0:1789760722.999780:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707282820414 time 707282559998 state 0x1 ref 1 00000100:00100000:0.0:1789760722.999803:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707282820414 time 707320519412 state 0x0 ref 1 00000100:00100000:0.0:1789760722.999816:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04931c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495762816:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760722.999865:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21889024, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa807 00010000:00010000:0.0:1789760722.999873:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545459ec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.999889:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760722.999893:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.999899:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545459ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.999913:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545459ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.025194:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972661 is committed 00002000:00010000:0.0:1789760723.025423:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21889024 -> 21893120 00010000:00010000:0.0:1789760723.025438:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545459ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.025452:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545459ec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.025462:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545459ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.025469:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760723.025474:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545459ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.025483:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545459ec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.025620:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545459ec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.025629:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545459ec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.025762:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495762816:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25943us (26158us total) trans 4294972661 rc 0/0 00000100:00100000:0.0:1789760723.025773:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5373 00000100:00100000:0.0:1789760723.025784:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707282820414 time 707320519412 state 0x2 ref 0 00000100:00100000:3.0:1789760723.031906:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760723.032023:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495762944 00000100:00100000:0.0:1789760723.032032:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495762944 00000100:00100000:0.0:1789760723.032152:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707282820414 time 707320519412 state 0x0 ref 1 00000100:00100000:0.0:1789760723.032192:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707320779828 time 707320519412 state 0x1 ref 1 00000100:00100000:0.0:1789760723.032223:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707320779828 time 707352939286 state 0x0 ref 1 00000100:00100000:0.0:1789760723.032237:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495762944:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760723.032320:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21893120, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa807 00010000:00010000:0.0:1789760723.032330:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545459f3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.032347:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760723.032351:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.032358:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545459f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.032373:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545459f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760723.039400:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760723.067664:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972662 is committed 00002000:00010000:2.0:1789760723.067766:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21893120 -> 21897216 00010000:00010000:2.0:1789760723.067782:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545459f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.067797:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545459f3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.067807:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545459f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.067814:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760723.067819:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545459f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.067828:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545459f3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.067850:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545459f3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.067857:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545459f3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.067956:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495762944:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35718us (36053us total) trans 4294972662 rc 0/0 00000100:00100000:2.0:1789760723.067967:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5374 00000100:00100000:2.0:1789760723.067977:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707320779828 time 707352939286 state 0x2 ref 0 00000100:00100000:3.0:1789760723.071152:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760723.071603:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495763072 00000100:00100000:2.0:1789760723.071613:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495763072 00000100:00100000:2.0:1789760723.071667:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707320779828 time 707352939286 state 0x0 ref 1 00000100:00100000:2.0:1789760723.071681:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707353199702 time 707352939286 state 0x1 ref 1 00000100:00100000:2.0:1789760723.071702:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707353199702 time 707392419270 state 0x0 ref 1 00000100:00100000:2.0:1789760723.071714:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495763072:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760723.071761:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21897216, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa807 00010000:00010000:2.0:1789760723.071770:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545459fa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.071785:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760723.071789:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.071796:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545459fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.071810:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545459fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760723.113625:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972663 is committed 00002000:00010000:2.0:1789760723.114021:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21897216 -> 21901312 00010000:00010000:2.0:1789760723.114037:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545459fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.114049:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545459fa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.114063:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545459fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.114070:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760723.114075:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545459fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.114084:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545459fa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.114102:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545459fa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.114109:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545459fa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.114445:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495763072:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42726us (43296us total) trans 4294972663 rc 0/0 00000100:00100000:2.0:1789760723.114456:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5375 00000100:00100000:2.0:1789760723.114467:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707353199702 time 707392419270 state 0x2 ref 0 00000100:00100000:1.0:1789760723.118096:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760723.118206:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495763200 00000100:00100000:0.0:1789760723.118215:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495763200 00000100:00100000:0.0:1789760723.118272:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707353199702 time 707392419270 state 0x0 ref 1 00000100:00100000:0.0:1789760723.118320:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707392679686 time 707392419270 state 0x1 ref 1 00000100:00100000:0.0:1789760723.118351:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707392679686 time 707439068467 state 0x0 ref 1 00000100:00100000:0.0:1789760723.118365:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495763200:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760723.118414:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21901312, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa807 00010000:00010000:0.0:1789760723.118423:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545a01 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.118440:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760723.118444:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.118450:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545a01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.118464:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545a01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760723.144836:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972664 is committed 00002000:00010000:0.0:1789760723.144948:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21901312 -> 21905408 00010000:00010000:0.0:1789760723.144965:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545a01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.144978:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545a01 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.144989:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545a01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.144997:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760723.145002:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545a01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.145013:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545a01 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.145037:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545a01 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.145045:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545a01 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.145191:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495763200:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26816us (27091us total) trans 4294972664 rc 0/0 00000100:00100000:0.0:1789760723.145203:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5376 00000100:00100000:0.0:1789760723.145213:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707392679686 time 707439068467 state 0x2 ref 0 00000100:00100000:0.0:1789760723.149940:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760723.150028:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495763328 00000100:00100000:2.0:1789760723.150038:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495763328 00000100:00100000:2.0:1789760723.150094:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707392679686 time 707439068467 state 0x0 ref 1 00000100:00100000:2.0:1789760723.150112:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707439328883 time 707439068467 state 0x1 ref 1 00000100:00100000:2.0:1789760723.150135:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707439328883 time 707470851900 state 0x0 ref 1 00000100:00100000:2.0:1789760723.150146:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495763328:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760723.150326:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21905408, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa807 00010000:00010000:2.0:1789760723.150336:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545a08 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.150352:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760723.150357:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.150363:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545a08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.150378:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545a08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760723.178564:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21905408 -> 21909504 00010000:00010000:0.0:1789760723.178582:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545a08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.178594:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545a08 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.178603:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545a08 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.178621:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760723.178626:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545a08 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.178634:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545a08 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.178653:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545a08 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.178664:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545a08 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.178755:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495763328:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28606us (28821us total) trans 4294972665 rc 0/0 00000100:00100000:0.0:1789760723.178766:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5377 00000100:00100000:0.0:1789760723.178777:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707439328883 time 707470851900 state 0x2 ref 0 00000001:00080000:3.0:1789760723.181652:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972665 is committed 00000100:00100000:2.0:1789760723.182476:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760723.182619:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495763456 00000100:00100000:3.0:1789760723.182629:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495763456 00000100:00100000:3.0:1789760723.182680:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707439328883 time 707470851900 state 0x0 ref 1 00000100:00100000:3.0:1789760723.182695:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707471112316 time 707470851900 state 0x1 ref 1 00000100:00100000:3.0:1789760723.182721:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707471112316 time 707503438611 state 0x0 ref 1 00000100:00100000:3.0:1789760723.182733:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495763456:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760723.182778:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21909504, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa807 00010000:00010000:3.0:1789760723.182787:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545a0f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.182804:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760723.182809:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760723.182814:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545a0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.182833:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545a0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760723.209058:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972666 is committed 00002000:00010000:3.0:1789760723.209130:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21909504 -> 21913600 00010000:00010000:3.0:1789760723.209145:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545a0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.209320:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545a0f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.209330:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545a0f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.209337:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760723.209342:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545a0f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.209353:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545a0f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.209377:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545a0f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.209385:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545a0f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760723.209484:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495763456:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26746us (27008us total) trans 4294972666 rc 0/0 00000100:00100000:3.0:1789760723.209496:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5378 00000100:00100000:3.0:1789760723.210013:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707471112316 time 707503438611 state 0x2 ref 0 00000100:00100000:3.0:1789760723.213759:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760723.213994:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495763584 00000100:00100000:0.0:1789760723.214005:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495763584 00000100:00100000:0.0:1789760723.214059:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707471112316 time 707503438611 state 0x0 ref 1 00000100:00100000:0.0:1789760723.214075:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707503699027 time 707503438611 state 0x1 ref 1 00000100:00100000:0.0:1789760723.214108:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707503699027 time 707534825417 state 0x0 ref 1 00000100:00100000:0.0:1789760723.214122:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495763584:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760723.214191:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21913600, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa807 00010000:00010000:0.0:1789760723.214201:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a16 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.214217:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760723.214222:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.214229:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.214246:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.254986:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972667 is committed 00002000:00010000:0.0:1789760723.255391:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21913600 -> 21917696 00010000:00010000:0.0:1789760723.255406:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.255420:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a16 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.255430:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a16 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.255438:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760723.255443:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a16 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.255452:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a16 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.255470:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a16 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.255478:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a16 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.255750:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495763584:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41626us (41996us total) trans 4294972667 rc 0/0 00000100:00100000:0.0:1789760723.255760:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5379 00000100:00100000:0.0:1789760723.255770:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707503699027 time 707534825417 state 0x2 ref 0 00000100:00100000:0.0:1789760723.259747:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760723.259860:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495763712 00000100:00100000:2.0:1789760723.259870:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495763712 00000100:00100000:2.0:1789760723.259925:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707503699027 time 707534825417 state 0x0 ref 1 00000100:00100000:2.0:1789760723.259940:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707535085833 time 707534825417 state 0x1 ref 1 00000100:00100000:2.0:1789760723.260058:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707535085833 time 707580773443 state 0x0 ref 1 00000100:00100000:2.0:1789760723.260075:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495763712:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760723.260129:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21917696, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa807 00010000:00010000:2.0:1789760723.260139:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545a1d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.260256:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760723.260261:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.260268:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545a1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.260408:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545a1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.290845:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972668 is committed 00002000:00010000:2.0:1789760723.290942:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21917696 -> 21921792 00010000:00010000:2.0:1789760723.290958:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545a1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.290971:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545a1d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.290979:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545a1d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.290986:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760723.290990:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545a1d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.291012:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545a1d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.291032:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545a1d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.291041:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545a1d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.291132:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495763712:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31054us (31389us total) trans 4294972668 rc 0/0 00000100:00100000:2.0:1789760723.291142:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5380 00000100:00100000:2.0:1789760723.291152:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707535085833 time 707580773443 state 0x2 ref 0 00000100:00100000:2.0:1789760723.295371:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760723.295653:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495763840 00000100:00100000:3.0:1789760723.295664:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495763840 00000100:00100000:3.0:1789760723.295727:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707535085833 time 707580773443 state 0x0 ref 1 00000100:00100000:3.0:1789760723.295750:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707581033859 time 707580773443 state 0x1 ref 1 00000100:00100000:3.0:1789760723.295780:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707581033859 time 707616497411 state 0x0 ref 1 00000100:00100000:3.0:1789760723.295793:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495763840:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760723.295846:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21921792, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa807 00010000:00010000:3.0:1789760723.295855:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545a24 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.295873:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760723.295878:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760723.295884:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545a24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.295898:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545a24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760723.323394:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972669 is committed 00002000:00010000:1.0:1789760723.323575:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21921792 -> 21925888 00010000:00010000:1.0:1789760723.323593:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545a24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.323668:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545a24 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.323679:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545a24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.323686:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760723.323691:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545a24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.323700:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545a24 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.323721:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545a24 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.323729:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545a24 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.323827:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495763840:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28030us (28459us total) trans 4294972669 rc 0/0 00000100:00100000:1.0:1789760723.323837:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5381 00000100:00100000:1.0:1789760723.323846:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707581033859 time 707616497411 state 0x2 ref 0 00000100:00100000:3.0:1789760723.327463:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760723.327604:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495763968 00000100:00100000:0.0:1789760723.327624:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495763968 00000100:00100000:0.0:1789760723.327678:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707581033859 time 707616497411 state 0x0 ref 1 00000100:00100000:0.0:1789760723.327693:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707616757827 time 707616497411 state 0x1 ref 1 00000100:00100000:0.0:1789760723.327720:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707616757827 time 707648437754 state 0x0 ref 1 00000100:00100000:0.0:1789760723.327732:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495763968:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760723.327780:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21925888, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa807 00010000:00010000:0.0:1789760723.327789:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a2b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.327804:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760723.327808:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.327815:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.327829:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760723.357167:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21925888 -> 21929984 00010000:00010000:0.0:1789760723.357192:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.357206:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a2b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.357216:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a2b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.357223:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760723.357228:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a2b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.357237:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a2b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.357256:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a2b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.357264:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a2b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.357377:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495763968:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29642us (29917us total) trans 4294972670 rc 0/0 00000100:00100000:0.0:1789760723.357390:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5382 00000100:00100000:0.0:1789760723.357401:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707616757827 time 707648437754 state 0x2 ref 0 00000001:00080000:3.0:1789760723.360535:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972670 is committed 00000100:00100000:0.0:1789760723.361067:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760723.361147:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495764096 00000100:00100000:2.0:1789760723.361168:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495764096 00000100:00100000:2.0:1789760723.361230:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707616757827 time 707648437754 state 0x0 ref 1 00000100:00100000:2.0:1789760723.361245:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707648698170 time 707648437754 state 0x1 ref 1 00000100:00100000:2.0:1789760723.361349:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707648698170 time 707682065425 state 0x0 ref 1 00000100:00100000:2.0:1789760723.361361:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903639d050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495764096:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760723.361409:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21929984, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa807 00010000:00010000:2.0:1789760723.361418:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545a32 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.361435:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760723.361440:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.361447:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545a32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.361461:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545a32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.387782:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972671 is committed 00002000:00010000:3.0:1789760723.387873:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21929984 -> 21934080 00010000:00010000:3.0:1789760723.387889:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545a32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.387902:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545a32 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.387911:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545a32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.387919:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760723.387924:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545a32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.387932:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545a32 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.387952:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545a32 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.387960:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545a32 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760723.388096:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903639d050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495764096:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26732us (27030us total) trans 4294972671 rc 0/0 00000100:00100000:3.0:1789760723.388107:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5383 00000100:00100000:3.0:1789760723.388119:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707648698170 time 707682065425 state 0x2 ref 0 00000100:00100000:2.0:1789760723.392634:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760723.392870:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495764224 00000100:00100000:1.0:1789760723.392883:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495764224 00000100:00100000:1.0:1789760723.392941:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707648698170 time 707682065425 state 0x0 ref 1 00000100:00100000:1.0:1789760723.392959:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707682325841 time 707682065425 state 0x1 ref 1 00000100:00100000:1.0:1789760723.393038:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707682325841 time 707713754316 state 0x0 ref 1 00000100:00100000:1.0:1789760723.393049:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495764224:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760723.393094:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21934080, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa807 00010000:00010000:1.0:1789760723.393104:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545a39 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.393118:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760723.393123:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.393129:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545a39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.393143:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545a39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760723.438013:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21934080 -> 21938176 00010000:00010000:0.0:1789760723.438034:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545a39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.438047:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545a39 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.438055:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545a39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.438062:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760723.438067:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545a39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.438075:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545a39 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.438094:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545a39 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.438102:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545a39 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.438220:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495764224:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45161us (45585us total) trans 4294972672 rc 0/0 00000100:00100000:0.0:1789760723.438231:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5384 00000100:00100000:0.0:1789760723.438240:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707682325841 time 707713754316 state 0x2 ref 0 00000001:00080000:1.0:1789760723.438647:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972672 is committed 00000100:00100000:2.0:1789760723.444122:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760723.444244:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495764352 00000100:00100000:0.0:1789760723.444254:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495764352 00000100:00100000:0.0:1789760723.444350:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707682325841 time 707713754316 state 0x0 ref 1 00000100:00100000:0.0:1789760723.444369:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707714014732 time 707713754316 state 0x1 ref 1 00000100:00100000:0.0:1789760723.444409:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707714014732 time 707765125187 state 0x0 ref 1 00000100:00100000:0.0:1789760723.444424:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495764352:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760723.444484:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21938176, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa807 00010000:00010000:0.0:1789760723.444507:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a40 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.444530:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760723.444536:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.444549:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.444565:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760723.453916:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760723.453956:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760723.453979:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760723.454005:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495764608 00000100:00100000:3.0:1789760723.454014:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495764608 00000100:00100000:3.0:1789760723.454068:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495764736 00000100:00100000:3.0:1789760723.454071:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495764736 00000100:00100000:3.0:1789760723.454096:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 465 00000100:00100000:3.0:1789760723.454106:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:17022965-1e4f-46fb-915c-61730096b8e2+8:2356:x1876699495764608:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:3.0:1789760723.454186:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:17022965-1e4f-46fb-915c-61730096b8e2+8:2356:x1876699495764608:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 70us (272us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760723.454194:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 465 00000100:00100000:3.0:1789760723.454211:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 466 00000100:00100000:3.0:1789760723.454217:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2356:x1876699495764736:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:3.0:1789760723.454242:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2356:x1876699495764736:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 26us (262us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760723.454249:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 466 00000100:00080000:3.0:1789760723.454268:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760723.456333:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495764480 00000100:00100000:0.0:1789760723.456353:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495764480 00000100:00100000:0.0:1789760723.456440:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 394 00000100:00100000:0.0:1789760723.456452:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929012316850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2356:x1876699495764480:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:0.0:1789760723.456579:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929012316850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2356:x1876699495764480:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 125us (2617us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760723.456587:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 394 00002000:00010000:2.0:1789760723.479076:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21938176 -> 21942272 00010000:00010000:2.0:1789760723.479096:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.479110:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a40 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.479120:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.479127:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760723.479133:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.479142:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a40 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.479172:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a40 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.479189:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a40 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.479312:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495764352:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34857us (35169us total) trans 4294972673 rc 0/0 00000100:00100000:2.0:1789760723.479323:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5385 00000100:00100000:2.0:1789760723.479333:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707714014732 time 707765125187 state 0x2 ref 0 00000001:00080000:2.0:1789760723.481012:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972673 is committed 00000100:00100000:3.0:1789760723.483330:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760723.483434:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495764864 00000100:00100000:2.0:1789760723.483445:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495764864 00000100:00100000:2.0:1789760723.483521:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707714014732 time 707765125187 state 0x0 ref 1 00000100:00100000:2.0:1789760723.483534:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707765385603 time 707765125187 state 0x1 ref 1 00000100:00100000:2.0:1789760723.483562:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707765385603 time 707804280021 state 0x0 ref 1 00000100:00100000:2.0:1789760723.483571:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495764864:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760723.483613:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21942272, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa807 00010000:00010000:2.0:1789760723.483620:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545a47 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.483635:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760723.483638:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.483643:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545a47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.483654:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545a47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760723.508306:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972674 is committed 00002000:00010000:0.0:1789760723.508493:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21942272 -> 21946368 00010000:00010000:0.0:1789760723.508588:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545a47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.508600:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545a47 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.508609:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545a47 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.508616:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760723.508621:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545a47 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.508629:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545a47 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.508648:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545a47 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.508657:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545a47 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.508785:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495764864:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25209us (25502us total) trans 4294972674 rc 0/0 00000100:00100000:0.0:1789760723.508796:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5386 00000100:00100000:0.0:1789760723.508807:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707765385603 time 707804280021 state 0x2 ref 0 00000100:00100000:0.0:1789760723.512263:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760723.512623:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495764992 00000100:00100000:1.0:1789760723.512635:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495764992 00000100:00100000:1.0:1789760723.512695:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707765385603 time 707804280021 state 0x0 ref 1 00000100:00100000:1.0:1789760723.512710:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707804540437 time 707804280021 state 0x1 ref 1 00000100:00100000:1.0:1789760723.512741:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707804540437 time 707833457561 state 0x0 ref 1 00000100:00100000:1.0:1789760723.512758:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495764992:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760723.512804:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21946368, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa807 00010000:00010000:1.0:1789760723.512812:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545a4e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.512827:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760723.512832:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.512840:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545a4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.512852:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545a4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760723.533683:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21946368 -> 21950464 00010000:00010000:0.0:1789760723.533704:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545a4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.533717:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545a4e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.533726:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545a4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.533734:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760723.533740:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545a4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.533749:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545a4e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.533774:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545a4e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.533783:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545a4e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.533884:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495764992:12345-192.168.203.9@tcp:4:dd.0 Request processed in 21123us (21627us total) trans 4294972675 rc 0/0 00000100:00100000:0.0:1789760723.533896:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5387 00000100:00100000:0.0:1789760723.533907:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707804540437 time 707833457561 state 0x2 ref 0 00000001:00080000:3.0:1789760723.537027:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972675 is committed 00000100:00100000:2.0:1789760723.537413:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760723.537488:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495765120 00000100:00100000:0.0:1789760723.537505:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495765120 00000100:00100000:0.0:1789760723.537561:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707804540437 time 707833457561 state 0x0 ref 1 00000100:00100000:0.0:1789760723.537575:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707833717977 time 707833457561 state 0x1 ref 1 00000100:00100000:0.0:1789760723.537610:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707833717977 time 707858325764 state 0x0 ref 1 00000100:00100000:0.0:1789760723.537621:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495765120:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760723.537667:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21950464, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa807 00010000:00010000:0.0:1789760723.537676:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545a55 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.537692:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760723.537699:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.537706:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545a55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.537720:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545a55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760723.563567:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972676 is committed 00002000:00010000:2.0:1789760723.563674:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21950464 -> 21954560 00010000:00010000:2.0:1789760723.563689:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545a55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.563703:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545a55 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.563713:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545a55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.563720:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760723.563783:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545a55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.563792:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545a55 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.563810:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545a55 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.563818:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545a55 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.563903:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495765120:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26279us (26490us total) trans 4294972676 rc 0/0 00000100:00100000:2.0:1789760723.563914:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5388 00000100:00100000:2.0:1789760723.563923:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707833717977 time 707858325764 state 0x2 ref 0 00000100:00100000:2.0:1789760723.567894:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760723.567972:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495765248 00000100:00100000:0.0:1789760723.567982:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495765248 00000100:00100000:0.0:1789760723.568037:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707833717977 time 707858325764 state 0x0 ref 1 00000100:00100000:0.0:1789760723.568053:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707858586180 time 707858325764 state 0x1 ref 1 00000100:00100000:0.0:1789760723.568087:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707858586180 time 707888803574 state 0x0 ref 1 00000100:00100000:0.0:1789760723.568101:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495765248:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760723.568150:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21954560, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa807 00010000:00010000:0.0:1789760723.568168:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545a5c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.568193:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760723.568197:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.568204:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545a5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.568218:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545a5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760723.591197:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972677 is committed 00002000:00010000:0.0:1789760723.591376:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21954560 -> 21958656 00010000:00010000:0.0:1789760723.591391:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545a5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.591404:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545a5c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.591418:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545a5c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.591426:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760723.591431:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545a5c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.591440:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545a5c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.591461:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545a5c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.591469:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545a5c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.591594:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495765248:12345-192.168.203.9@tcp:4:dd.0 Request processed in 23490us (23703us total) trans 4294972677 rc 0/0 00000100:00100000:0.0:1789760723.591607:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5389 00000100:00100000:0.0:1789760723.591617:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707858586180 time 707888803574 state 0x2 ref 0 00000100:00100000:0.0:1789760723.596062:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760723.596225:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495765376 00000100:00100000:1.0:1789760723.596234:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495765376 00000100:00100000:1.0:1789760723.596311:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707858586180 time 707888803574 state 0x0 ref 1 00000100:00100000:1.0:1789760723.596326:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707889063990 time 707888803574 state 0x1 ref 1 00000100:00100000:1.0:1789760723.596347:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707889063990 time 707917064586 state 0x0 ref 1 00000100:00100000:1.0:1789760723.596359:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495765376:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760723.596404:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21958656, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa807 00010000:00010000:1.0:1789760723.596413:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545a63 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.596435:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760723.596440:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.596446:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545a63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.596461:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545a63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.641719:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972678 is committed 00002000:00010000:0.0:1789760723.641800:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21958656 -> 21962752 00010000:00010000:0.0:1789760723.641817:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545a63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.641830:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545a63 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.641840:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545a63 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.641848:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760723.641853:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545a63 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.641864:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545a63 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.641895:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545a63 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.641903:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545a63 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.642010:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495765376:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45647us (45953us total) trans 4294972678 rc 0/0 00000100:00100000:0.0:1789760723.642022:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5390 00000100:00100000:0.0:1789760723.642032:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707889063990 time 707917064586 state 0x2 ref 0 00000100:00100000:2.0:1789760723.645534:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760723.645642:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495765504 00000100:00100000:0.0:1789760723.645652:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495765504 00000100:00100000:0.0:1789760723.645705:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707889063990 time 707917064586 state 0x0 ref 1 00000100:00100000:0.0:1789760723.645720:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707917325002 time 707917064586 state 0x1 ref 1 00000100:00100000:0.0:1789760723.645741:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707917325002 time 707966457931 state 0x0 ref 1 00000100:00100000:0.0:1789760723.645753:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495765504:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760723.645800:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21962752, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa807 00010000:00010000:0.0:1789760723.645809:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545a6a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.645826:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760723.645830:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.645837:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545a6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.645850:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545a6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.688959:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972679 is committed 00002000:00010000:0.0:1789760723.689063:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21962752 -> 21966848 00002000:00010000:0.0:1789760723.689068:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760722 -> 1789760723 00002000:00010000:0.0:1789760723.689070:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760722 -> 1789760723 00010000:00010000:0.0:1789760723.689082:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545a6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.689094:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545a6a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.689103:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545a6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.689110:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760723.689116:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545a6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.689132:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545a6a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.689373:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545a6a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.689383:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545a6a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.689510:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495765504:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43754us (43979us total) trans 4294972679 rc 0/0 00000100:00100000:0.0:1789760723.689520:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5391 00000100:00100000:0.0:1789760723.689529:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707917325002 time 707966457931 state 0x2 ref 0 00000100:00100000:3.0:1789760723.698677:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760723.698761:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495765632 00000100:00100000:0.0:1789760723.698772:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495765632 00000100:00100000:0.0:1789760723.698827:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707917325002 time 707966457931 state 0x0 ref 1 00000100:00100000:0.0:1789760723.698844:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707966718347 time 707966457931 state 0x1 ref 1 00000100:00100000:0.0:1789760723.698878:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707966718347 time 708019594543 state 0x0 ref 1 00000100:00100000:0.0:1789760723.698891:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495765632:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760723.698937:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21966848, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa807 00010000:00010000:0.0:1789760723.698946:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545a71 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.698962:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760723.698967:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.698973:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545a71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.698984:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545a71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760723.735842:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972680 is committed 00002000:00010000:1.0:1789760723.735943:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21966848 -> 21970944 00010000:00010000:1.0:1789760723.735957:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545a71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.735970:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545a71 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.735979:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545a71 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.735987:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760723.735992:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545a71 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.736001:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545a71 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.736025:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545a71 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.736033:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545a71 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.736127:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495765632:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37233us (37454us total) trans 4294972680 rc 0/0 00000100:00100000:1.0:1789760723.736138:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5392 00000100:00100000:1.0:1789760723.736149:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707966718347 time 708019594543 state 0x2 ref 0 00000100:00100000:1.0:1789760723.749275:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760723.749405:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495765760 00000100:00100000:0.0:1789760723.749416:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495765760 00000100:00100000:0.0:1789760723.749466:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 707966718347 time 708019594543 state 0x0 ref 1 00000100:00100000:0.0:1789760723.749478:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708019854959 time 708019594543 state 0x1 ref 1 00000100:00100000:0.0:1789760723.749503:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708019854959 time 708070220185 state 0x0 ref 1 00000100:00100000:0.0:1789760723.749516:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495765760:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760723.749574:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21970944, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa807 00010000:00010000:0.0:1789760723.749584:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a78 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.749599:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760723.749604:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.749610:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.749621:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760723.793962:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972681 is committed 00002000:00010000:0.0:1789760723.794276:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21970944 -> 21975040 00010000:00010000:0.0:1789760723.794311:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.794329:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a78 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.794338:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a78 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.794346:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760723.794351:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a78 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.794360:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a78 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.794379:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a78 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.794387:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a78 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.794479:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495765760:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44961us (45209us total) trans 4294972681 rc 0/0 00000100:00100000:0.0:1789760723.794489:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5393 00000100:00100000:0.0:1789760723.794499:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708019854959 time 708070220185 state 0x2 ref 0 00000100:00100000:0.0:1789760723.797552:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760723.797696:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495765888 00000100:00100000:2.0:1789760723.797706:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495765888 00000100:00100000:2.0:1789760723.797760:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708019854959 time 708070220185 state 0x0 ref 1 00000100:00100000:2.0:1789760723.797774:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708070480601 time 708070220185 state 0x1 ref 1 00000100:00100000:2.0:1789760723.797811:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708070480601 time 708118527482 state 0x0 ref 1 00000100:00100000:2.0:1789760723.797824:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495765888:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760723.797872:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21975040, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa807 00010000:00010000:2.0:1789760723.797881:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545a7f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.797897:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760723.797901:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.797908:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545a7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.797920:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545a7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.806456:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760723.826139:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21975040 -> 21979136 00010000:00010000:1.0:1789760723.826182:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545a7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.826196:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545a7f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.826205:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545a7f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.826213:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760723.826217:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545a7f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.826226:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545a7f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.826250:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545a7f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.826257:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545a7f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.826387:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495765888:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28561us (28839us total) trans 4294972682 rc 0/0 00000100:00100000:1.0:1789760723.826397:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5394 00000100:00100000:1.0:1789760723.826407:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708070480601 time 708118527482 state 0x2 ref 0 00000001:00080000:0.0:1789760723.826927:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972682 is committed 00000100:00100000:3.0:1789760723.832164:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760723.832257:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495766016 00000100:00100000:1.0:1789760723.832267:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495766016 00000100:00100000:1.0:1789760723.832359:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708070480601 time 708118527482 state 0x0 ref 1 00000100:00100000:1.0:1789760723.832374:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708118787898 time 708118527482 state 0x1 ref 1 00000100:00100000:1.0:1789760723.832393:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708118787898 time 708153111101 state 0x0 ref 1 00000100:00100000:1.0:1789760723.832401:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495766016:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760723.832441:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21979136, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa807 00010000:00010000:1.0:1789760723.832448:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545a86 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.832461:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760723.832464:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.832469:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545a86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.832480:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545a86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760723.866872:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972683 is committed 00002000:00010000:0.0:1789760723.866889:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21979136 -> 21983232 00010000:00010000:0.0:1789760723.866904:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545a86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.866918:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545a86 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.866933:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545a86 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.866940:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760723.866945:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545a86 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.866955:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545a86 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.866975:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545a86 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.866984:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545a86 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.867248:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495766016:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34840us (35088us total) trans 4294972683 rc 0/0 00000100:00100000:0.0:1789760723.867260:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5395 00000100:00100000:0.0:1789760723.867270:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708118787898 time 708153111101 state 0x2 ref 0 00000100:00100000:1.0:1789760723.872022:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760723.872108:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495766144 00000100:00100000:0.0:1789760723.872118:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495766144 00000100:00100000:0.0:1789760723.872185:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708118787898 time 708153111101 state 0x0 ref 1 00000100:00100000:0.0:1789760723.872201:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708153371517 time 708153111101 state 0x1 ref 1 00000100:00100000:0.0:1789760723.872229:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708153371517 time 708192946412 state 0x0 ref 1 00000100:00100000:0.0:1789760723.872241:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495766144:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760723.872334:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21983232, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa807 00010000:00010000:0.0:1789760723.872344:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a8d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.872360:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760723.872365:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.872371:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.872385:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760723.928375:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972684 is committed 00002000:00010000:2.0:1789760723.929364:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21983232 -> 21987328 00010000:00010000:2.0:1789760723.929382:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.929394:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a8d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.929404:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a8d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.929411:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760723.929416:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a8d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.929426:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a8d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.929450:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a8d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.929458:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545a8d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.929552:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495766144:12345-192.168.203.9@tcp:4:dd.0 Request processed in 57308us (57535us total) trans 4294972684 rc 0/0 00000100:00100000:2.0:1789760723.929563:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5396 00000100:00100000:2.0:1789760723.929573:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708153371517 time 708192946412 state 0x2 ref 0 00000100:00100000:2.0:1789760723.942817:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760723.948327:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495766272 00000100:00100000:1.0:1789760723.948342:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495766272 00000100:00100000:1.0:1789760723.948403:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708153371517 time 708192946412 state 0x0 ref 1 00000100:00100000:1.0:1789760723.948418:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708193206828 time 708192946412 state 0x1 ref 1 00000100:00100000:1.0:1789760723.948513:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708193206828 time 708269229119 state 0x0 ref 1 00000100:00100000:1.0:1789760723.948537:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fdcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495766272:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760723.948586:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21987328, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa807 00010000:00010000:1.0:1789760723.948596:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354545a94 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.948613:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760723.948618:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.948624:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354545a94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.948639:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354545a94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760723.993320:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972685 is committed 00002000:00010000:2.0:1789760723.997310:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21987328 -> 21991424 00010000:00010000:2.0:1789760723.997329:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354545a94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.997343:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354545a94 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.997352:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354545a94 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.997360:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760723.997365:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354545a94 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.997379:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354545a94 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.997398:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354545a94 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.997406:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354545a94 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.997504:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fdcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495766272:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48966us (54691us total) trans 4294972685 rc 0/0 00000100:00100000:2.0:1789760723.997513:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5397 00000100:00100000:2.0:1789760723.997523:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708193206828 time 708269229119 state 0x2 ref 0 00000100:00100000:3.0:1789760724.003230:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760724.005511:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495766400 00000100:00100000:0.0:1789760724.005524:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495766400 00000100:00100000:0.0:1789760724.005761:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708193206828 time 708269229119 state 0x0 ref 1 00000100:00100000:0.0:1789760724.005781:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708269489535 time 708269229119 state 0x1 ref 1 00000100:00100000:1.0:1789760724.005976:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708269489535 time 708326690293 state 0x0 ref 1 00000100:00100000:1.0:1789760724.005991:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495766400:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760724.006037:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21991424, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa807 00010000:00010000:1.0:1789760724.006045:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545a9b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.006061:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760724.006066:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760724.006072:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545a9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.006085:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545a9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760724.046060:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972686 is committed 00002000:00010000:0.0:1789760724.046187:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21991424 -> 21995520 00010000:00010000:0.0:1789760724.046204:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545a9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.046217:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545a9b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.046225:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545a9b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.046233:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760724.046237:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545a9b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.046245:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545a9b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.046264:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545a9b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.046271:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545a9b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760724.046418:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495766400:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40426us (43192us total) trans 4294972686 rc 0/0 00000100:00100000:0.0:1789760724.046429:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5398 00000100:00100000:0.0:1789760724.046439:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708269489535 time 708326690293 state 0x2 ref 0 00000100:00100000:3.0:1789760724.049252:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760724.049677:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495766528 00000100:00100000:2.0:1789760724.049686:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495766528 00000100:00100000:3.0:1789760724.049867:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708269489535 time 708326690293 state 0x0 ref 1 00000100:00100000:3.0:1789760724.049892:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708326950709 time 708326690293 state 0x1 ref 1 00000100:00100000:3.0:1789760724.049954:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708326950709 time 708370670117 state 0x0 ref 1 00000100:00100000:3.0:1789760724.049969:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495766528:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760724.050018:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21995520, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa807 00010000:00010000:3.0:1789760724.050028:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545aa2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.050045:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760724.050050:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760724.050057:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545aa2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.050071:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545aa2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760724.100611:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972687 is committed 00002000:00010000:3.0:1789760724.100751:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21995520 -> 21999616 00010000:00010000:3.0:1789760724.100768:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545aa2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.100781:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545aa2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.100792:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545aa2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.100799:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760724.100804:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545aa2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.100813:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545aa2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.100833:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545aa2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.100841:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545aa2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760724.100943:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495766528:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50973us (51695us total) trans 4294972687 rc 0/0 00000100:00100000:3.0:1789760724.100953:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5399 00000100:00100000:3.0:1789760724.100963:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708326950709 time 708370670117 state 0x2 ref 0 00000100:00100000:2.0:1789760724.103796:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760724.103901:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495766656 00000100:00100000:1.0:1789760724.103910:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495766656 00000100:00100000:1.0:1789760724.103966:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708326950709 time 708370670117 state 0x0 ref 1 00000100:00100000:1.0:1789760724.103981:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708370930533 time 708370670117 state 0x1 ref 1 00000100:00100000:1.0:1789760724.104012:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708370930533 time 708424728172 state 0x0 ref 1 00000100:00100000:1.0:1789760724.104024:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fdc9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495766656:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760724.104070:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21999616, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa807 00010000:00010000:1.0:1789760724.104079:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545aa9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.104094:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760724.104098:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760724.104104:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545aa9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.104117:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545aa9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760724.174315:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972688 is committed 00002000:00010000:2.0:1789760724.174422:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21999616 -> 22003712 00010000:00010000:2.0:1789760724.174438:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545aa9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.174450:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545aa9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.174464:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545aa9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.174472:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760724.174476:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545aa9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.174485:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545aa9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.174503:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545aa9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.174511:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545aa9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.174603:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fdc9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495766656:12345-192.168.203.9@tcp:4:dd.0 Request processed in 70576us (70812us total) trans 4294972688 rc 0/0 00000100:00100000:2.0:1789760724.174613:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5400 00000100:00100000:2.0:1789760724.174624:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708370930533 time 708424728172 state 0x2 ref 0 00000100:00100000:2.0:1789760724.177833:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760724.177903:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495766784 00000100:00100000:0.0:1789760724.177912:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495766784 00000100:00100000:0.0:1789760724.177961:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708370930533 time 708424728172 state 0x0 ref 1 00000100:00100000:0.0:1789760724.177974:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708424988588 time 708424728172 state 0x1 ref 1 00000100:00100000:0.0:1789760724.178000:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708424988588 time 708498717230 state 0x0 ref 1 00000100:00100000:0.0:1789760724.178011:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495766784:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760724.178054:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22003712, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa807 00010000:00010000:0.0:1789760724.178062:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ab0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.178078:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760724.178082:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760724.178088:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ab0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.178101:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ab0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760724.204958:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22003712 -> 22007808 00000001:00080000:2.0:1789760724.204961:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972689 is committed 00010000:00010000:0.0:1789760724.204979:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ab0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.204993:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ab0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.205002:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ab0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.205011:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760724.205016:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ab0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.205028:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ab0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.205048:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ab0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.205056:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ab0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760724.205143:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495766784:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27128us (27312us total) trans 4294972689 rc 0/0 00000100:00100000:0.0:1789760724.205166:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5401 00000100:00100000:0.0:1789760724.205189:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708424988588 time 708498717230 state 0x2 ref 0 00000100:00100000:1.0:1789760724.209090:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760724.209192:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495766912 00000100:00100000:3.0:1789760724.209202:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495766912 00000100:00100000:3.0:1789760724.209254:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708424988588 time 708498717230 state 0x0 ref 1 00000100:00100000:3.0:1789760724.209269:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708498977646 time 708498717230 state 0x1 ref 1 00000100:00100000:3.0:1789760724.209330:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708498977646 time 708530044929 state 0x0 ref 1 00000100:00100000:2.0:1789760724.209431:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495766912:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760724.209482:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22007808, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa807 00010000:00010000:2.0:1789760724.209492:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545ab7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.209516:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760724.209520:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.209526:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545ab7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.209539:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545ab7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760724.243052:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972690 is committed 00002000:00010000:2.0:1789760724.243304:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22007808 -> 22011904 00010000:00010000:2.0:1789760724.243319:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545ab7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.243332:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545ab7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.243342:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545ab7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.243349:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760724.243354:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545ab7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.243364:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545ab7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.243389:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545ab7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.243397:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545ab7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.243491:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495766912:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34145us (34403us total) trans 4294972690 rc 0/0 00000100:00100000:2.0:1789760724.243502:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5402 00000100:00100000:2.0:1789760724.243513:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708498977646 time 708530044929 state 0x2 ref 0 00000100:00100000:2.0:1789760724.247770:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760724.247905:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495767040 00000100:00100000:1.0:1789760724.247916:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495767040 00000100:00100000:1.0:1789760724.247971:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708498977646 time 708530044929 state 0x0 ref 1 00000100:00100000:1.0:1789760724.247986:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708530305345 time 708530044929 state 0x1 ref 1 00000100:00100000:1.0:1789760724.248018:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708530305345 time 708568734557 state 0x0 ref 1 00000100:00100000:1.0:1789760724.248030:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495767040:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760724.248077:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22011904, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa807 00010000:00010000:1.0:1789760724.248086:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545abe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.248102:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760724.248106:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760724.248112:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545abe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.248127:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545abe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760724.283093:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972691 is committed 00002000:00010000:2.0:1789760724.283196:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22011904 -> 22016000 00010000:00010000:2.0:1789760724.283212:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545abe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.283226:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545abe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.283234:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545abe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.283241:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760724.283246:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545abe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.283261:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545abe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.283298:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545abe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.283307:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545abe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.283435:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495767040:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35402us (35670us total) trans 4294972691 rc 0/0 00000100:00100000:2.0:1789760724.283445:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5403 00000100:00100000:2.0:1789760724.283455:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708530305345 time 708568734557 state 0x2 ref 0 00000100:00100000:3.0:1789760724.291701:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760724.291831:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495767168 00000100:00100000:0.0:1789760724.291842:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495767168 00000100:00100000:0.0:1789760724.291894:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708530305345 time 708568734557 state 0x0 ref 1 00000100:00100000:0.0:1789760724.291921:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708568994973 time 708568734557 state 0x1 ref 1 00000100:00100000:0.0:1789760724.292002:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708568994973 time 708612718275 state 0x0 ref 1 00000100:00100000:0.0:1789760724.292015:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495767168:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760724.292064:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22016000, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa807 00010000:00010000:0.0:1789760724.292078:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ac5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.292094:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760724.292099:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760724.292105:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ac5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.292119:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ac5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760724.325806:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972692 is committed 00002000:00010000:0.0:1789760724.325888:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22016000 -> 22020096 00010000:00010000:0.0:1789760724.325903:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ac5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.325915:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ac5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.325929:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ac5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.325936:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760724.325941:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ac5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.325950:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ac5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.325969:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ac5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.325977:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ac5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760724.326102:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495767168:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34085us (34407us total) trans 4294972692 rc 0/0 00000100:00100000:0.0:1789760724.326122:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5404 00000100:00100000:0.0:1789760724.326132:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708568994973 time 708612718275 state 0x2 ref 0 00000100:00100000:1.0:1789760724.330340:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760724.330513:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495767296 00000100:00100000:2.0:1789760724.330524:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495767296 00000100:00100000:2.0:1789760724.330580:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708568994973 time 708612718275 state 0x0 ref 1 00000100:00100000:2.0:1789760724.330597:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708612978691 time 708612718275 state 0x1 ref 1 00000100:00100000:2.0:1789760724.330627:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708612978691 time 708651343629 state 0x0 ref 1 00000100:00100000:2.0:1789760724.330640:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92901237ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495767296:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760724.330693:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22020096, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa807 00010000:00010000:2.0:1789760724.330702:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545acc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.330720:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760724.330724:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.330731:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545acc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.330745:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545acc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760724.402167:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22020096 -> 22024192 00002000:00010000:2.0:1789760724.402187:0:13055:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 43015 -> 45063 00010000:00010000:2.0:1789760724.402201:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545acc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.402215:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545acc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.402224:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545acc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.402362:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760724.402369:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545acc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.402381:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545acc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.402409:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545acc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.402417:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545acc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.402508:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92901237ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495767296:12345-192.168.203.9@tcp:4:dd.0 Request processed in 71867us (72175us total) trans 4294972693 rc 0/0 00000100:00100000:2.0:1789760724.402517:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5405 00000100:00100000:2.0:1789760724.402527:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708612978691 time 708651343629 state 0x2 ref 0 00000001:00080000:0.0:1789760724.402639:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972693 is committed 00000100:00100000:3.0:1789760724.406256:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760724.406376:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495767424 00000100:00100000:2.0:1789760724.406386:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495767424 00000100:00100000:2.0:1789760724.406440:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708612978691 time 708651343629 state 0x0 ref 1 00000100:00100000:2.0:1789760724.406455:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708651604045 time 708651343629 state 0x1 ref 1 00000100:00100000:2.0:1789760724.406538:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708651604045 time 708727254675 state 0x0 ref 1 00000100:00100000:2.0:1789760724.406548:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495767424:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760724.406587:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22024192, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb007 00010000:00010000:2.0:1789760724.406594:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ad3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.406606:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760724.406610:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.406614:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ad3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.406624:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ad3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760724.433481:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972694 is committed 00002000:00010000:2.0:1789760724.434991:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22024192 -> 22028288 00010000:00010000:2.0:1789760724.435011:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ad3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.435024:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ad3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.435069:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ad3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.435076:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760724.435081:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ad3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.435089:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ad3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.435109:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ad3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.435116:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ad3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.435318:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495767424:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28723us (29086us total) trans 4294972694 rc 0/0 00000100:00100000:2.0:1789760724.435331:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5406 00000100:00100000:2.0:1789760724.435341:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708651604045 time 708727254675 state 0x2 ref 0 00000100:00100000:2.0:1789760724.445879:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760724.446039:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495767552 00000100:00100000:0.0:1789760724.446050:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495767552 00000100:00100000:0.0:1789760724.446104:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708651604045 time 708727254675 state 0x0 ref 1 00000100:00100000:0.0:1789760724.446120:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708727515091 time 708727254675 state 0x1 ref 1 00000100:00100000:0.0:1789760724.446197:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708727515091 time 708766913477 state 0x0 ref 1 00000100:00100000:0.0:1789760724.446211:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495767552:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760724.446258:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22028288, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb007 00010000:00010000:0.0:1789760724.446329:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ada lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.446346:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760724.446351:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760724.446358:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ada lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.446372:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ada lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760724.509981:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972695 is committed 00002000:00010000:3.0:1789760724.510122:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22028288 -> 22032384 00010000:00010000:3.0:1789760724.510139:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ada lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.510225:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ada lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.510235:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ada lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.510243:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760724.510249:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ada lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.510257:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ada lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.510437:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ada lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.510446:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545ada lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.512041:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495767552:12345-192.168.203.9@tcp:4:dd.0 Request processed in 65824us (66162us total) trans 4294972695 rc 0/0 00000100:00100000:2.0:1789760724.512056:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5407 00000100:00100000:2.0:1789760724.512066:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708727515091 time 708766913477 state 0x2 ref 0 00000100:00100000:1.0:1789760724.524792:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760724.524899:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495767680 00000100:00100000:2.0:1789760724.524909:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495767680 00000100:00100000:2.0:1789760724.524961:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708727515091 time 708766913477 state 0x0 ref 1 00000100:00100000:2.0:1789760724.524975:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708767173893 time 708766913477 state 0x1 ref 1 00000100:00100000:2.0:1789760724.525005:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708767173893 time 708845721777 state 0x0 ref 1 00000100:00100000:2.0:1789760724.525017:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495767680:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760724.525060:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22032384, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb007 00010000:00010000:2.0:1789760724.525069:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ae1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.525085:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760724.525091:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.525097:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ae1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.525110:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ae1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760724.588456:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972696 is committed 00002000:00010000:2.0:1789760724.588622:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22032384 -> 22036480 00010000:00010000:2.0:1789760724.588639:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ae1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.588651:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ae1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.588660:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ae1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.588669:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760724.588674:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ae1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.588684:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ae1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.588705:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ae1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.588717:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ae1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.588818:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495767680:12345-192.168.203.9@tcp:4:dd.0 Request processed in 63797us (64029us total) trans 4294972696 rc 0/0 00000100:00100000:2.0:1789760724.588830:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5408 00000100:00100000:2.0:1789760724.588841:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708767173893 time 708845721777 state 0x2 ref 0 00000100:00100000:0.0:1789760724.592061:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760724.592141:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495767808 00000100:00100000:2.0:1789760724.592159:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495767808 00000100:00100000:2.0:1789760724.592219:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708767173893 time 708845721777 state 0x0 ref 1 00000100:00100000:2.0:1789760724.592234:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708845982193 time 708845721777 state 0x1 ref 1 00000100:00100000:2.0:1789760724.592266:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708845982193 time 708912983543 state 0x0 ref 1 00000100:00100000:2.0:1789760724.592322:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495767808:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760724.592371:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22036480, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb007 00010000:00010000:2.0:1789760724.592381:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545ae8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.592395:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760724.592400:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.592406:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545ae8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.592425:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545ae8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760724.628516:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972697 is committed 00002000:00010000:3.0:1789760724.628754:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22036480 -> 22040576 00010000:00010000:3.0:1789760724.628769:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545ae8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.628781:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545ae8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.628790:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545ae8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.628797:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760724.628802:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545ae8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.628811:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545ae8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.628833:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545ae8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.628845:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545ae8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760724.628937:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495767808:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36655us (36882us total) trans 4294972697 rc 0/0 00000100:00100000:3.0:1789760724.628948:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5409 00000100:00100000:3.0:1789760724.628958:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708845982193 time 708912983543 state 0x2 ref 0 00000100:00100000:2.0:1789760724.635379:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760724.635901:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495767936 00000100:00100000:0.0:1789760724.635913:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495767936 00000100:00100000:0.0:1789760724.635966:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708845982193 time 708912983543 state 0x0 ref 1 00000100:00100000:0.0:1789760724.635981:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708913243959 time 708912983543 state 0x1 ref 1 00000100:00100000:0.0:1789760724.636065:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708913243959 time 708956781386 state 0x0 ref 1 00000100:00100000:0.0:1789760724.636077:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495767936:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760724.636126:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22040576, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb007 00010000:00010000:0.0:1789760724.636134:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545aef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.636160:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760724.636173:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760724.636181:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545aef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.636194:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545aef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760724.668376:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972698 is committed 00002000:00010000:3.0:1789760724.668495:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22040576 -> 22044672 00002000:00010000:3.0:1789760724.668501:0:6714:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760723 -> 1789760724 00002000:00010000:3.0:1789760724.668504:0:6714:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760723 -> 1789760724 00010000:00010000:3.0:1789760724.668518:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545aef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.668531:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545aef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.668540:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545aef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.668547:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760724.668552:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545aef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.668561:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545aef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.668581:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545aef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.668589:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545aef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760724.668678:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495767936:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32598us (33305us total) trans 4294972698 rc 0/0 00000100:00100000:3.0:1789760724.668688:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5410 00000100:00100000:3.0:1789760724.668698:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708913243959 time 708956781386 state 0x2 ref 0 00000100:00100000:0.0:1789760724.680612:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760724.680821:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495768064 00000100:00100000:2.0:1789760724.680830:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495768064 00000100:00100000:2.0:1789760724.680884:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708913243959 time 708956781386 state 0x0 ref 1 00000100:00100000:2.0:1789760724.680900:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708957041802 time 708956781386 state 0x1 ref 1 00000100:00100000:2.0:1789760724.680931:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708957041802 time 709001647019 state 0x0 ref 1 00000100:00100000:2.0:1789760724.680944:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495768064:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760724.680990:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22044672, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:2.0:1789760724.681000:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545af6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.681016:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760724.681021:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.681028:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545af6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.681043:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545af6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760724.746835:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972699 is committed 00002000:00010000:2.0:1789760724.749387:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22044672 -> 22048768 00010000:00010000:2.0:1789760724.749409:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545af6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.749435:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545af6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.749444:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545af6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.749451:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760724.749456:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545af6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.749465:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545af6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.749486:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545af6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.749493:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545af6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.749588:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495768064:12345-192.168.203.9@tcp:4:dd.0 Request processed in 68642us (68982us total) trans 4294972699 rc 0/0 00000100:00100000:2.0:1789760724.749601:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5411 00000100:00100000:2.0:1789760724.749611:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708957041802 time 709001647019 state 0x2 ref 0 00000100:00100000:0.0:1789760724.758129:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760724.758240:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495768192 00000100:00100000:3.0:1789760724.758249:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495768192 00000100:00100000:3.0:1789760724.758473:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 708957041802 time 709001647019 state 0x0 ref 1 00000100:00100000:3.0:1789760724.758490:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709001907435 time 709001647019 state 0x1 ref 1 00000100:00100000:3.0:1789760724.758573:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709001907435 time 709079288211 state 0x0 ref 1 00000100:00100000:3.0:1789760724.758587:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03095500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495768192:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760724.758636:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22048768, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:3.0:1789760724.758645:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545afd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.758667:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760724.758672:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760724.758679:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545afd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.758692:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545afd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760724.802949:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972700 is committed 00002000:00010000:3.0:1789760724.803273:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22048768 -> 22052864 00010000:00010000:3.0:1789760724.803305:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545afd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.803315:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545afd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.803321:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545afd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.803325:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760724.803328:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545afd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.803334:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545afd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.803348:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545afd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.803353:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545afd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760724.803422:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495768192:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44835us (45299us total) trans 4294972700 rc 0/0 00000100:00100000:3.0:1789760724.803429:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5412 00000100:00100000:3.0:1789760724.803436:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709001907435 time 709079288211 state 0x2 ref 0 00000100:00100000:0.0:1789760724.807531:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760724.807656:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495768320 00000100:00100000:3.0:1789760724.807665:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495768320 00000100:00100000:3.0:1789760724.807717:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709001907435 time 709079288211 state 0x0 ref 1 00000100:00100000:3.0:1789760724.807732:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709079548627 time 709079288211 state 0x1 ref 1 00000100:00100000:3.0:1789760724.807762:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709079548627 time 709128478658 state 0x0 ref 1 00000100:00100000:3.0:1789760724.807775:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03097b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495768320:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760724.807821:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22052864, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:3.0:1789760724.807830:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b04 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.807846:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760724.807851:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760724.807858:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.807876:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.830496:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760724.834980:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972701 is committed 00002000:00010000:3.0:1789760724.835080:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22052864 -> 22056960 00010000:00010000:3.0:1789760724.835094:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.835107:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b04 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.835117:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b04 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.835124:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760724.835129:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b04 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.835138:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b04 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.835226:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b04 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.835234:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b04 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760724.835425:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03097b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495768320:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27648us (27898us total) trans 4294972701 rc 0/0 00000100:00100000:3.0:1789760724.835436:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5413 00000100:00100000:3.0:1789760724.835445:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709079548627 time 709128478658 state 0x2 ref 0 00000100:00100000:0.0:1789760724.839727:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760724.839837:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495768448 00000100:00100000:2.0:1789760724.839847:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495768448 00000100:00100000:2.0:1789760724.839915:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709079548627 time 709128478658 state 0x0 ref 1 00000100:00100000:2.0:1789760724.839933:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709128739074 time 709128478658 state 0x1 ref 1 00000100:00100000:2.0:1789760724.839966:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709128739074 time 709160682594 state 0x0 ref 1 00000100:00100000:2.0:1789760724.839978:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03094700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495768448:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760724.840027:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22056960, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:2.0:1789760724.840038:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545b0b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.840055:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760724.840060:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.840067:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545b0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.840082:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545b0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760724.866848:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972702 is committed 00002000:00010000:0.0:1789760724.866961:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22056960 -> 22061056 00010000:00010000:0.0:1789760724.866977:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545b0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.866995:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545b0b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.867004:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545b0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.867011:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760724.867015:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545b0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.867025:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545b0b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.867048:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545b0b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.867056:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545b0b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760724.867356:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03094700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495768448:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27374us (27634us total) trans 4294972702 rc 0/0 00000100:00100000:0.0:1789760724.867367:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5414 00000100:00100000:0.0:1789760724.867377:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709128739074 time 709160682594 state 0x2 ref 0 00000100:00100000:0.0:1789760724.869969:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760724.870068:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495768576 00000100:00100000:3.0:1789760724.870077:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495768576 00000100:00100000:3.0:1789760724.870125:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709128739074 time 709160682594 state 0x0 ref 1 00000100:00100000:3.0:1789760724.870139:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709160943010 time 709160682594 state 0x1 ref 1 00000100:00100000:3.0:1789760724.870379:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709160943010 time 709191093676 state 0x0 ref 1 00000100:00100000:3.0:1789760724.870392:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03095180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495768576:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760724.870436:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22061056, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:3.0:1789760724.870444:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b12 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.870460:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760724.870464:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760724.870470:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.870484:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760724.897855:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972703 is committed 00002000:00010000:3.0:1789760724.897999:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22061056 -> 22065152 00010000:00010000:3.0:1789760724.898014:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.898028:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b12 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.898037:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.898045:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760724.898050:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.898072:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b12 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.898094:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b12 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.898102:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b12 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760724.898876:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495768576:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28479us (28908us total) trans 4294972703 rc 0/0 00000100:00100000:3.0:1789760724.898890:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5415 00000100:00100000:3.0:1789760724.898901:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709160943010 time 709191093676 state 0x2 ref 0 00000100:00100000:2.0:1789760724.902883:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760724.903001:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495768704 00000100:00100000:3.0:1789760724.903011:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495768704 00000100:00100000:3.0:1789760724.903062:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709160943010 time 709191093676 state 0x0 ref 1 00000100:00100000:3.0:1789760724.903077:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709191354092 time 709191093676 state 0x1 ref 1 00000100:00100000:3.0:1789760724.903107:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709191354092 time 709223824322 state 0x0 ref 1 00000100:00100000:3.0:1789760724.903119:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495768704:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760724.903187:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22065152, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:3.0:1789760724.903195:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b19 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.903211:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760724.903215:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760724.903221:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.903234:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760724.931849:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972704 is committed 00002000:00010000:3.0:1789760724.931952:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22065152 -> 22069248 00010000:00010000:3.0:1789760724.931967:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.931981:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b19 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.931990:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.931997:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760724.932002:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.932010:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b19 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.932030:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b19 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.932037:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b19 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760724.932180:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495768704:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29049us (29294us total) trans 4294972704 rc 0/0 00000100:00100000:3.0:1789760724.932190:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5416 00000100:00100000:3.0:1789760724.932200:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709191354092 time 709223824322 state 0x2 ref 0 00000100:00100000:2.0:1789760724.939048:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760724.939226:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495768832 00000100:00100000:0.0:1789760724.939236:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495768832 00000100:00100000:0.0:1789760724.939474:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709191354092 time 709223824322 state 0x0 ref 1 00000100:00100000:0.0:1789760724.939502:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709224084738 time 709223824322 state 0x1 ref 1 00000100:00100000:0.0:1789760724.939534:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709224084738 time 709260250760 state 0x0 ref 1 00000100:00100000:0.0:1789760724.939547:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495768832:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760724.939596:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22069248, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:0.0:1789760724.939605:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545b20 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.939621:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760724.939626:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760724.939633:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545b20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.939646:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545b20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760724.984341:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972705 is committed 00002000:00010000:3.0:1789760724.984453:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22069248 -> 22073344 00010000:00010000:3.0:1789760724.984469:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545b20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.984481:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545b20 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.984500:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545b20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.984508:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760724.984514:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545b20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.984523:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545b20 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.984543:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545b20 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.984551:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545b20 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760724.984640:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495768832:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45090us (45597us total) trans 4294972705 rc 0/0 00000100:00100000:3.0:1789760724.984650:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5417 00000100:00100000:3.0:1789760724.984659:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709224084738 time 709260250760 state 0x2 ref 0 00000100:00100000:1.0:1789760724.987658:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760724.987815:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495768960 00000100:00100000:3.0:1789760724.987825:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495768960 00000100:00100000:3.0:1789760724.987875:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709224084738 time 709260250760 state 0x0 ref 1 00000100:00100000:3.0:1789760724.987889:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709260511176 time 709260250760 state 0x1 ref 1 00000100:00100000:3.0:1789760724.987917:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709260511176 time 709308634747 state 0x0 ref 1 00000100:00100000:3.0:1789760724.987929:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495768960:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760724.987975:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22073344, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:3.0:1789760724.987984:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b27 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.988000:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760724.988004:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760724.988011:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.988024:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760725.033806:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22073344 -> 22077440 00010000:00010000:3.0:1789760725.033829:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.033841:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b27 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.033850:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b27 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.033858:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760725.033862:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b27 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.033870:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b27 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.033889:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b27 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.033898:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b27 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.033991:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495768960:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46058us (46335us total) trans 4294972706 rc 0/0 00000100:00100000:3.0:1789760725.034001:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5418 00000100:00100000:3.0:1789760725.034012:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709260511176 time 709308634747 state 0x2 ref 0 00000001:00080000:1.0:1789760725.037548:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972706 is committed 00000100:00100000:2.0:1789760725.039431:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760725.039518:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495769088 00000100:00100000:3.0:1789760725.039529:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495769088 00000100:00100000:3.0:1789760725.039699:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709260511176 time 709308634747 state 0x0 ref 1 00000100:00100000:3.0:1789760725.039715:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709308895163 time 709308634747 state 0x1 ref 1 00000100:00100000:3.0:1789760725.039749:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709308895163 time 709360465787 state 0x0 ref 1 00000100:00100000:3.0:1789760725.039761:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495769088:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760725.039807:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22077440, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:3.0:1789760725.039817:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b2e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.039833:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760725.039837:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.039843:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.039857:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760725.040254:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789760725.102098:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22077440 -> 22081536 00010000:00010000:3.0:1789760725.102119:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.102133:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b2e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760725.102242:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972707 is committed 00010000:00010000:3.0:1789760725.102302:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.102313:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760725.102318:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.102327:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b2e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.102347:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b2e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.102355:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b2e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.102617:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495769088:12345-192.168.203.9@tcp:4:dd.0 Request processed in 62853us (63189us total) trans 4294972707 rc 0/0 00000100:00100000:3.0:1789760725.102628:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5419 00000100:00100000:3.0:1789760725.102639:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709308895163 time 709360465787 state 0x2 ref 0 00000100:00100000:2.0:1789760725.109358:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760725.110716:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495769216 00000100:00100000:3.0:1789760725.110727:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495769216 00000100:00100000:3.0:1789760725.110782:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709308895163 time 709360465787 state 0x0 ref 1 00000100:00100000:3.0:1789760725.110797:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709360726203 time 709360465787 state 0x1 ref 1 00000100:00100000:3.0:1789760725.110832:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709360726203 time 709431548944 state 0x0 ref 1 00000100:00100000:3.0:1789760725.110846:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495769216:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760725.110891:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22081536, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:3.0:1789760725.110900:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b35 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.110917:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760725.110922:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.110929:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.110944:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760725.148917:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972708 is committed 00002000:00010000:2.0:1789760725.149397:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22081536 -> 22085632 00010000:00010000:2.0:1789760725.149417:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.149431:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b35 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.149441:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b35 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.149448:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760725.149454:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b35 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.149463:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b35 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.149482:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b35 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.149490:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545b35 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760725.149581:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495769216:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38734us (40240us total) trans 4294972708 rc 0/0 00000100:00100000:2.0:1789760725.149592:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5420 00000100:00100000:2.0:1789760725.149603:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709360726203 time 709431548944 state 0x2 ref 0 00000100:00100000:1.0:1789760725.155612:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760725.155686:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495769344 00000100:00100000:3.0:1789760725.155696:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495769344 00000100:00100000:3.0:1789760725.155804:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709360726203 time 709431548944 state 0x0 ref 1 00000100:00100000:3.0:1789760725.155820:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709431809360 time 709431548944 state 0x1 ref 1 00000100:00100000:3.0:1789760725.155853:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709431809360 time 709476569477 state 0x0 ref 1 00000100:00100000:3.0:1789760725.155866:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370bd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495769344:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760725.155914:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22085632, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:3.0:1789760725.155922:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b3c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.155938:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760725.155942:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.155950:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.155962:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760725.202258:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972709 is committed 00002000:00010000:3.0:1789760725.202668:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22085632 -> 22089728 00010000:00010000:3.0:1789760725.202683:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.202697:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b3c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.202707:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b3c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.202715:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760725.202720:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b3c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.202728:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b3c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.202747:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b3c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.202756:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b3c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.202848:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370bd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495769344:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46980us (47240us total) trans 4294972709 rc 0/0 00000100:00100000:3.0:1789760725.202859:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5421 00000100:00100000:3.0:1789760725.202869:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709431809360 time 709476569477 state 0x2 ref 0 00000100:00100000:0.0:1789760725.209900:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760725.210027:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495769472 00000100:00100000:3.0:1789760725.210050:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495769472 00000100:00100000:3.0:1789760725.210106:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709431809360 time 709476569477 state 0x0 ref 1 00000100:00100000:3.0:1789760725.210122:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709476829893 time 709476569477 state 0x1 ref 1 00000100:00100000:3.0:1789760725.210153:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709476829893 time 709530870000 state 0x0 ref 1 00000100:00100000:3.0:1789760725.210183:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495769472:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760725.210229:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22089728, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:3.0:1789760725.210238:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b43 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.210256:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760725.210260:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.210266:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.210436:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760725.249005:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972710 is committed 00002000:00010000:0.0:1789760725.249105:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22089728 -> 22093824 00010000:00010000:0.0:1789760725.249121:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.249134:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b43 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.249155:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b43 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.249171:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760725.249176:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b43 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.249185:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b43 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.249205:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b43 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.249213:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545b43 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760725.249341:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495769472:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39173us (39455us total) trans 4294972710 rc 0/0 00000100:00100000:0.0:1789760725.249352:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5422 00000100:00100000:0.0:1789760725.249361:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709476829893 time 709530870000 state 0x2 ref 0 00000100:00100000:2.0:1789760725.257126:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760725.257776:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495769600 00000100:00100000:3.0:1789760725.257788:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495769600 00000100:00100000:2.0:1789760725.257990:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709476829893 time 709530870000 state 0x0 ref 1 00000100:00100000:2.0:1789760725.258008:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709531130416 time 709530870000 state 0x1 ref 1 00000100:00100000:2.0:1789760725.258079:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709531130416 time 709578794583 state 0x0 ref 1 00000100:00100000:2.0:1789760725.258092:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495769600:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760725.258150:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22093824, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:2.0:1789760725.258159:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545b4a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.258185:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760725.258190:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.258197:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545b4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.258211:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545b4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760725.278436:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9290404a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520303872:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760725.278509:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760725.278580:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520303872 00000100:00100000:0.0:1789760725.278588:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520303872 00000100:00100000:0.0:1789760725.278647:0:6711:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 193 00000100:00100000:0.0:1789760725.278655:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520303872:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789760725.278678:0:6711:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527892480 pending 1703936 free 3835691008 avail 3821010944 00000100:00100000:0.0:1789760725.278719:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520303872:12345-0@lo:13:osp-pre-0-0.0 Request processed in 64us (210us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760725.278727:0:6711:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 193 00000100:00080000:0.0:1789760725.278750:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760725.278785:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9290404a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520303872:0@lo:13:osp-pre-0-0.0 00000001:00080000:0.0:1789760725.293313:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972711 is committed 00002000:00010000:2.0:1789760725.293492:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22093824 -> 22097920 00010000:00010000:2.0:1789760725.293514:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545b4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.293540:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545b4a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.293549:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545b4a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.293557:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760725.293563:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545b4a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.293572:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545b4a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.293718:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545b4a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.293729:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545b4a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760725.293830:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495769600:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35732us (36706us total) trans 4294972711 rc 0/0 00000100:00100000:2.0:1789760725.293846:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5423 00000100:00100000:2.0:1789760725.293856:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709531130416 time 709578794583 state 0x2 ref 0 00000100:00100000:1.0:1789760725.298105:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760725.298233:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495769728 00000100:00100000:3.0:1789760725.298245:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495769728 00000100:00100000:3.0:1789760725.298337:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709531130416 time 709578794583 state 0x0 ref 1 00000100:00100000:3.0:1789760725.298354:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709579054999 time 709578794583 state 0x1 ref 1 00000100:00100000:3.0:1789760725.298385:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709579054999 time 709619101569 state 0x0 ref 1 00000100:00100000:3.0:1789760725.298398:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495769728:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760725.298447:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22097920, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:3.0:1789760725.298456:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b51 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.298474:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760725.298479:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.298485:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.298499:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760725.325497:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22097920 -> 22102016 00010000:00010000:3.0:1789760725.325517:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.325530:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b51 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.325539:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b51 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.325546:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760725.325551:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b51 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.325562:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b51 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.325579:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b51 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.325588:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b51 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760725.325644:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972712 is committed 00000100:00100000:3.0:1789760725.325687:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495769728:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27288us (27590us total) trans 4294972712 rc 0/0 00000100:00100000:3.0:1789760725.325698:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5424 00000100:00100000:3.0:1789760725.325708:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709579054999 time 709619101569 state 0x2 ref 0 00000100:00100000:0.0:1789760725.340424:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760725.340534:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495769856 00000100:00100000:3.0:1789760725.340545:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495769856 00000100:00100000:3.0:1789760725.340605:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709579054999 time 709619101569 state 0x0 ref 1 00000100:00100000:3.0:1789760725.340624:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709619361985 time 709619101569 state 0x1 ref 1 00000100:00100000:3.0:1789760725.340658:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709619361985 time 709661374527 state 0x0 ref 1 00000100:00100000:3.0:1789760725.340669:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495769856:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760725.340719:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22102016, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:3.0:1789760725.340729:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b58 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.340745:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760725.340749:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.340756:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.340771:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760725.387855:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972713 is committed 00002000:00010000:3.0:1789760725.388212:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22102016 -> 22106112 00010000:00010000:3.0:1789760725.388227:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.388239:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b58 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.388252:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.388260:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760725.388265:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.388274:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b58 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.388334:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b58 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.388342:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b58 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.388431:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495769856:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47759us (48013us total) trans 4294972713 rc 0/0 00000100:00100000:3.0:1789760725.388976:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5425 00000100:00100000:3.0:1789760725.388987:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709619361985 time 709661374527 state 0x2 ref 0 00000100:00100000:1.0:1789760725.399474:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760725.399616:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495769984 00000100:00100000:2.0:1789760725.399626:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495769984 00000100:00100000:2.0:1789760725.399682:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709619361985 time 709661374527 state 0x0 ref 1 00000100:00100000:2.0:1789760725.399697:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709661634943 time 709661374527 state 0x1 ref 1 00000100:00100000:2.0:1789760725.399727:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709661634943 time 709720444274 state 0x0 ref 1 00000100:00100000:2.0:1789760725.399739:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495769984:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760725.399785:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22106112, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:2.0:1789760725.399795:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545b5f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.399813:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760725.399818:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.399824:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545b5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.399838:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545b5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760725.453948:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22106112 -> 22110208 00010000:00010000:0.0:1789760725.453969:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545b5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.453983:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545b5f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.453993:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545b5f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.454000:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760725.454004:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545b5f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.454013:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545b5f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.454037:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545b5f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.454045:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545b5f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760725.454136:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495769984:12345-192.168.203.9@tcp:4:dd.0 Request processed in 54394us (54679us total) trans 4294972714 rc 0/0 00000100:00100000:0.0:1789760725.454156:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5426 00000100:00100000:0.0:1789760725.454175:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709661634943 time 709720444274 state 0x2 ref 0 00000001:00080000:1.0:1789760725.454312:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972714 is committed 00000100:00100000:0.0:1789760725.457710:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760725.457845:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495770112 00000100:00100000:3.0:1789760725.457854:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495770112 00000100:00100000:3.0:1789760725.457911:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709661634943 time 709720444274 state 0x0 ref 1 00000100:00100000:3.0:1789760725.457927:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709720704690 time 709720444274 state 0x1 ref 1 00000100:00100000:3.0:1789760725.457959:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709720704690 time 709778675648 state 0x0 ref 1 00000100:00100000:3.0:1789760725.457972:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495770112:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760725.458017:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22110208, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:3.0:1789760725.458026:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545b66 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.458043:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760725.458047:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.458053:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545b66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.458067:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545b66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760725.498499:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972715 is committed 00002000:00010000:0.0:1789760725.499149:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22110208 -> 22114304 00010000:00010000:0.0:1789760725.499175:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545b66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.499188:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545b66 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.499197:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545b66 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.499205:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760725.499209:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545b66 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.499224:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545b66 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.499243:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545b66 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.499371:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545b66 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760725.499469:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495770112:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41495us (41763us total) trans 4294972715 rc 0/0 00000100:00100000:0.0:1789760725.499478:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5427 00000100:00100000:0.0:1789760725.499488:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709720704690 time 709778675648 state 0x2 ref 0 00000100:00100000:3.0:1789760725.502811:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760725.503066:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495770240 00000100:00100000:0.0:1789760725.503077:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495770240 00000100:00100000:0.0:1789760725.503130:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709720704690 time 709778675648 state 0x0 ref 1 00000100:00100000:0.0:1789760725.503146:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709778936064 time 709778675648 state 0x1 ref 1 00000100:00100000:0.0:1789760725.503187:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709778936064 time 709823903942 state 0x0 ref 1 00000100:00100000:0.0:1789760725.503201:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495770240:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760725.503257:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22114304, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:0.0:1789760725.503266:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545b6d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.503315:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760725.503320:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760725.503327:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545b6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.503340:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545b6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760725.541300:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972716 is committed 00002000:00010000:3.0:1789760725.541979:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22114304 -> 22118400 00010000:00010000:3.0:1789760725.541996:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545b6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.542009:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545b6d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.542018:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545b6d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.542032:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760725.542037:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545b6d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.542047:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545b6d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.542065:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545b6d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.542073:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545b6d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.542157:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495770240:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38953us (39351us total) trans 4294972716 rc 0/0 00000100:00100000:3.0:1789760725.542179:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5428 00000100:00100000:3.0:1789760725.542189:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709778936064 time 709823903942 state 0x2 ref 0 00000100:00100000:1.0:1789760725.547565:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760725.547650:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495770368 00000100:00100000:0.0:1789760725.547659:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495770368 00000100:00100000:0.0:1789760725.547710:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709778936064 time 709823903942 state 0x0 ref 1 00000100:00100000:0.0:1789760725.547725:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709824164358 time 709823903942 state 0x1 ref 1 00000100:00100000:0.0:1789760725.547759:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709824164358 time 709868476153 state 0x0 ref 1 00000100:00100000:0.0:1789760725.547771:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495770368:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760725.547925:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22118400, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:0.0:1789760725.547934:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545b74 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.547949:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760725.547953:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760725.547958:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545b74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.547973:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545b74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760725.590638:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972717 is committed 00002000:00010000:3.0:1789760725.590768:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22118400 -> 22122496 00010000:00010000:3.0:1789760725.590784:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545b74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.590797:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545b74 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.591356:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545b74 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.591368:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760725.591373:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545b74 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.591383:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545b74 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.591402:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545b74 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.591410:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545b74 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.591494:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495770368:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43721us (43934us total) trans 4294972717 rc 0/0 00000100:00100000:3.0:1789760725.591504:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5429 00000100:00100000:3.0:1789760725.591513:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709824164358 time 709868476153 state 0x2 ref 0 00000100:00100000:0.0:1789760725.597811:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760725.597896:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495770496 00000100:00100000:3.0:1789760725.597905:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495770496 00000100:00100000:3.0:1789760725.597957:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709824164358 time 709868476153 state 0x0 ref 1 00000100:00100000:3.0:1789760725.597971:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709868736569 time 709868476153 state 0x1 ref 1 00000100:00100000:3.0:1789760725.598004:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709868736569 time 709918720467 state 0x0 ref 1 00000100:00100000:3.0:1789760725.598016:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495770496:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760725.598065:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22122496, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:3.0:1789760725.598074:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b7b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.598089:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760725.598093:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.598099:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.598112:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760725.625323:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972718 is committed 00002000:00010000:3.0:1789760725.627022:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22122496 -> 22126592 00010000:00010000:3.0:1789760725.627043:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.627056:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b7b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.627064:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.627071:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760725.627076:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.627086:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b7b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.627106:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b7b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.627113:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b7b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.627232:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495770496:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29213us (29424us total) trans 4294972718 rc 0/0 00000100:00100000:3.0:1789760725.627244:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5430 00000100:00100000:3.0:1789760725.627255:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709868736569 time 709918720467 state 0x2 ref 0 00000100:00100000:0.0:1789760725.638416:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760725.638668:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495770624 00000100:00100000:3.0:1789760725.638677:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495770624 00000100:00100000:3.0:1789760725.638729:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709868736569 time 709918720467 state 0x0 ref 1 00000100:00100000:3.0:1789760725.638743:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709918980883 time 709918720467 state 0x1 ref 1 00000100:00100000:3.0:1789760725.638767:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709918980883 time 709959483821 state 0x0 ref 1 00000100:00100000:3.0:1789760725.638778:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495770624:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760725.638825:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22126592, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:3.0:1789760725.638834:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b82 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.638848:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760725.638852:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.638857:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.638870:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760725.671016:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22126592 -> 22130688 00002000:00010000:3.0:1789760725.671025:0:6714:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760724 -> 1789760725 00002000:00010000:3.0:1789760725.671027:0:6714:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760724 -> 1789760725 00010000:00010000:3.0:1789760725.671040:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.671054:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b82 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.671064:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b82 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.671073:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760725.671086:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b82 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.671102:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b82 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.671121:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b82 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.671129:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b82 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.671235:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495770624:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32452us (32820us total) trans 4294972719 rc 0/0 00000100:00100000:3.0:1789760725.671246:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5431 00000100:00100000:3.0:1789760725.671256:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709918980883 time 709959483821 state 0x2 ref 0 00000001:00080000:2.0:1789760725.671833:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972719 is committed 00000100:00100000:3.0:1789760725.674578:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760725.674712:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495770752 00000100:00100000:0.0:1789760725.674723:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495770752 00000100:00100000:0.0:1789760725.674791:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709918980883 time 709959483821 state 0x0 ref 1 00000100:00100000:0.0:1789760725.674806:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709959744237 time 709959483821 state 0x1 ref 1 00000100:00100000:0.0:1789760725.674838:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709959744237 time 709995554949 state 0x0 ref 1 00000100:00100000:0.0:1789760725.674851:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495770752:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760725.674899:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22130688, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:0.0:1789760725.674908:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545b89 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.674923:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760725.674928:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760725.674934:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545b89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.674948:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545b89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760725.720030:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972720 is committed 00002000:00010000:3.0:1789760725.720115:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22130688 -> 22134784 00010000:00010000:3.0:1789760725.720130:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545b89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.720217:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545b89 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.720226:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545b89 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.720378:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760725.720385:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545b89 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.720394:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545b89 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.720414:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545b89 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.720421:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545b89 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760725.720733:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495770752:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45879us (46158us total) trans 4294972720 rc 0/0 00000100:00100000:2.0:1789760725.720743:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5432 00000100:00100000:2.0:1789760725.720752:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709959744237 time 709995554949 state 0x2 ref 0 00000100:00100000:0.0:1789760725.724014:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760725.724405:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495770880 00000100:00100000:3.0:1789760725.724416:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495770880 00000100:00100000:3.0:1789760725.724471:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709959744237 time 709995554949 state 0x0 ref 1 00000100:00100000:3.0:1789760725.724485:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709995815365 time 709995554949 state 0x1 ref 1 00000100:00100000:3.0:1789760725.724515:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709995815365 time 710045232855 state 0x0 ref 1 00000100:00100000:3.0:1789760725.724527:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495770880:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760725.724581:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22134784, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:3.0:1789760725.724662:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b90 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.724687:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760725.724692:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.724699:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.724714:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760725.763455:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972721 is committed 00002000:00010000:0.0:1789760725.763561:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22134784 -> 22138880 00010000:00010000:0.0:1789760725.763577:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.763590:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b90 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.763599:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.763606:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760725.763611:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.763619:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b90 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.763639:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b90 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.763647:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545b90 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760725.763736:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495770880:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39207us (39726us total) trans 4294972721 rc 0/0 00000100:00100000:0.0:1789760725.763748:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5433 00000100:00100000:0.0:1789760725.763758:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709995815365 time 710045232855 state 0x2 ref 0 00000100:00100000:0.0:1789760725.775916:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760725.777338:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495771008 00000100:00100000:3.0:1789760725.777351:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495771008 00000100:00100000:3.0:1789760725.777406:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 709995815365 time 710045232855 state 0x0 ref 1 00000100:00100000:3.0:1789760725.777420:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710045493271 time 710045232855 state 0x1 ref 1 00000100:00100000:3.0:1789760725.777455:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710045493271 time 710098172076 state 0x0 ref 1 00000100:00100000:3.0:1789760725.777467:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495771008:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760725.777512:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22138880, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:3.0:1789760725.777521:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545b97 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.777536:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760725.777541:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.777548:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545b97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.777566:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545b97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760725.842958:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972722 is committed 00002000:00010000:1.0:1789760725.843140:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22138880 -> 22142976 00010000:00010000:1.0:1789760725.843156:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545b97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.843176:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545b97 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.843186:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545b97 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.843194:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760725.843198:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545b97 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.843213:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545b97 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.843237:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545b97 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.843246:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545b97 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760725.843384:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495771008:12345-192.168.203.9@tcp:4:dd.0 Request processed in 65913us (67472us total) trans 4294972722 rc 0/0 00000100:00100000:1.0:1789760725.843394:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5434 00000100:00100000:1.0:1789760725.843403:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710045493271 time 710098172076 state 0x2 ref 0 00000100:00100000:2.0:1789760725.846369:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760725.846509:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495771136 00000100:00100000:0.0:1789760725.846522:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495771136 00000100:00100000:0.0:1789760725.846576:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710045493271 time 710098172076 state 0x0 ref 1 00000100:00100000:0.0:1789760725.846592:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710098432492 time 710098172076 state 0x1 ref 1 00000100:00100000:0.0:1789760725.846622:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710098432492 time 710167338920 state 0x0 ref 1 00000100:00100000:0.0:1789760725.846634:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495771136:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760725.846681:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22142976, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:0.0:1789760725.846690:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545b9e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.846707:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760725.846711:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760725.846717:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545b9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.846731:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545b9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.854493:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760725.891726:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972723 is committed 00002000:00010000:0.0:1789760725.892014:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22142976 -> 22147072 00010000:00010000:0.0:1789760725.892030:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545b9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.892042:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545b9e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.892050:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545b9e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.892056:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760725.892061:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545b9e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.892068:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545b9e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.892087:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545b9e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.892093:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545b9e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760725.892267:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495771136:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45629us (45903us total) trans 4294972723 rc 0/0 00000100:00100000:0.0:1789760725.893175:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5435 00000100:00100000:0.0:1789760725.893191:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710098432492 time 710167338920 state 0x2 ref 0 00000100:00100000:3.0:1789760725.898855:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760725.898967:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495771264 00000100:00100000:0.0:1789760725.898979:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495771264 00000100:00100000:0.0:1789760725.899038:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710098432492 time 710167338920 state 0x0 ref 1 00000100:00100000:0.0:1789760725.899055:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710167599336 time 710167338920 state 0x1 ref 1 00000100:00100000:0.0:1789760725.899084:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710167599336 time 710219801007 state 0x0 ref 1 00000100:00100000:0.0:1789760725.899096:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495771264:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760725.899153:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22147072, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:0.0:1789760725.899170:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545ba5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.899187:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760725.899192:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760725.899199:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545ba5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.899214:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545ba5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760725.932050:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972724 is committed 00002000:00010000:0.0:1789760725.932209:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22147072 -> 22151168 00010000:00010000:0.0:1789760725.932226:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545ba5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.932239:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545ba5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.932250:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545ba5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.932257:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760725.932262:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545ba5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.932270:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545ba5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.932306:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545ba5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.932315:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545ba5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760725.932398:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495771264:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33299us (33549us total) trans 4294972724 rc 0/0 00000100:00100000:0.0:1789760725.932407:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5436 00000100:00100000:0.0:1789760725.932417:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710167599336 time 710219801007 state 0x2 ref 0 00000100:00100000:3.0:1789760725.939051:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760725.939339:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495771392 00000100:00100000:1.0:1789760725.939352:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495771392 00000100:00100000:1.0:1789760725.939410:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710167599336 time 710219801007 state 0x0 ref 1 00000100:00100000:1.0:1789760725.939425:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710220061423 time 710219801007 state 0x1 ref 1 00000100:00100000:1.0:1789760725.939456:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710220061423 time 710260172867 state 0x0 ref 1 00000100:00100000:1.0:1789760725.939468:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495771392:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760725.939525:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22151168, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:1.0:1789760725.939534:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545bac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.939549:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760725.939553:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760725.939559:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545bac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.939572:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545bac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760725.976367:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972725 is committed 00002000:00010000:1.0:1789760725.976468:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22151168 -> 22155264 00010000:00010000:1.0:1789760725.976551:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545bac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.976565:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545bac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.976575:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545bac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.976582:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760725.976587:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545bac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.976596:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545bac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.976616:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545bac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.976627:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545bac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760725.976766:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495771392:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37295us (37720us total) trans 4294972725 rc 0/0 00000100:00100000:1.0:1789760725.976777:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5437 00000100:00100000:1.0:1789760725.976787:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710220061423 time 710260172867 state 0x2 ref 0 00000100:00100000:1.0:1789760725.986655:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760725.986756:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495771520 00000100:00100000:0.0:1789760725.986766:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495771520 00000100:00100000:0.0:1789760725.986818:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710220061423 time 710260172867 state 0x0 ref 1 00000100:00100000:0.0:1789760725.986838:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710260433283 time 710260172867 state 0x1 ref 1 00000100:00100000:0.0:1789760725.986870:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710260433283 time 710307587254 state 0x0 ref 1 00000100:00100000:0.0:1789760725.986882:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495771520:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760725.986927:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22155264, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:0.0:1789760725.986935:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545bb3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.986963:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760725.986967:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760725.986974:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545bb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.986987:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545bb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760726.019112:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972726 is committed 00002000:00010000:1.0:1789760726.019385:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22155264 -> 22159360 00010000:00010000:1.0:1789760726.019401:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545bb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.019415:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545bb3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.019424:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545bb3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.019431:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760726.019435:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545bb3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.019444:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545bb3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.019466:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545bb3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.019474:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545bb3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760726.019630:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495771520:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32745us (32980us total) trans 4294972726 rc 0/0 00000100:00100000:1.0:1789760726.019640:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5438 00000100:00100000:1.0:1789760726.019650:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710260433283 time 710307587254 state 0x2 ref 0 00000100:00100000:3.0:1789760726.022547:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760726.022813:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495771648 00000100:00100000:0.0:1789760726.022824:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495771648 00000100:00100000:0.0:1789760726.022886:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710260433283 time 710307587254 state 0x0 ref 1 00000100:00100000:0.0:1789760726.022902:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710307847670 time 710307587254 state 0x1 ref 1 00000100:00100000:0.0:1789760726.022938:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710307847670 time 710343654261 state 0x0 ref 1 00000100:00100000:0.0:1789760726.022950:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495771648:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760726.023003:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22159360, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:0.0:1789760726.023012:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545bba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.023030:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760726.023034:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.023040:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545bba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.023057:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545bba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760726.083514:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972727 is committed 00002000:00010000:0.0:1789760726.083590:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22159360 -> 22163456 00010000:00010000:0.0:1789760726.083605:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545bba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.083618:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545bba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.083628:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545bba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.083635:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760726.083641:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545bba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.083655:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545bba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.083675:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545bba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.083683:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545bba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.083770:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495771648:12345-192.168.203.9@tcp:4:dd.0 Request processed in 60816us (61229us total) trans 4294972727 rc 0/0 00000100:00100000:0.0:1789760726.083781:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5439 00000100:00100000:0.0:1789760726.083790:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710307847670 time 710343654261 state 0x2 ref 0 00000100:00100000:3.0:1789760726.087065:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760726.087115:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495771776 00000100:00100000:1.0:1789760726.087126:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495771776 00000100:00100000:1.0:1789760726.087255:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710307847670 time 710343654261 state 0x0 ref 1 00000100:00100000:1.0:1789760726.087271:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710343914677 time 710343654261 state 0x1 ref 1 00000100:00100000:1.0:1789760726.087344:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710343914677 time 710408059695 state 0x0 ref 1 00000100:00100000:1.0:1789760726.087358:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903639e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495771776:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760726.087410:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22163456, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:1.0:1789760726.087419:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545bc1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.087434:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760726.087438:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760726.087445:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545bc1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.087458:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545bc1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760726.142690:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972728 is committed 00002000:00010000:0.0:1789760726.144434:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22163456 -> 22167552 00010000:00010000:0.0:1789760726.144454:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545bc1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.144467:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545bc1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.145023:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545bc1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.145035:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760726.145040:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545bc1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.145049:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545bc1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.145068:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545bc1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.145076:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545bc1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.145430:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903639e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495771776:12345-192.168.203.9@tcp:4:dd.0 Request processed in 58069us (58366us total) trans 4294972728 rc 0/0 00000100:00100000:0.0:1789760726.145441:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5440 00000100:00100000:0.0:1789760726.145450:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710343914677 time 710408059695 state 0x2 ref 0 00000100:00100000:3.0:1789760726.149683:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760726.149756:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495771904 00000100:00100000:1.0:1789760726.149766:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495771904 00000100:00100000:1.0:1789760726.149822:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710343914677 time 710408059695 state 0x0 ref 1 00000100:00100000:1.0:1789760726.149836:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710408320111 time 710408059695 state 0x1 ref 1 00000100:00100000:1.0:1789760726.149869:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710408320111 time 710470586363 state 0x0 ref 1 00000100:00100000:1.0:1789760726.149882:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495771904:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760726.149926:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22167552, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:1.0:1789760726.149935:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545bc8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.149950:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760726.149955:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760726.149961:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545bc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.149975:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545bc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760726.186829:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972729 is committed 00002000:00010000:0.0:1789760726.186941:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22167552 -> 22171648 00010000:00010000:0.0:1789760726.186959:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545bc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.186972:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545bc8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.186982:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545bc8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.186996:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760726.187001:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545bc8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.187010:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545bc8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.187028:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545bc8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.187037:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545bc8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.187131:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495771904:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37243us (37448us total) trans 4294972729 rc 0/0 00000100:00100000:0.0:1789760726.187149:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5441 00000100:00100000:0.0:1789760726.187169:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710408320111 time 710470586363 state 0x2 ref 0 00000100:00100000:1.0:1789760726.191832:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760726.191912:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495772032 00000100:00100000:0.0:1789760726.191923:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495772032 00000100:00100000:0.0:1789760726.191978:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710408320111 time 710470586363 state 0x0 ref 1 00000100:00100000:0.0:1789760726.191994:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710470846779 time 710470586363 state 0x1 ref 1 00000100:00100000:0.0:1789760726.192027:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710470846779 time 710512743721 state 0x0 ref 1 00000100:00100000:0.0:1789760726.192042:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04292a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495772032:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760726.192085:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22171648, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:0.0:1789760726.192094:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545bcf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.192111:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760726.192116:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.192122:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545bcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.192146:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545bcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760726.224891:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22171648 -> 22175744 00010000:00010000:0.0:1789760726.224910:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545bcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.224923:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545bcf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.224932:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545bcf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.224939:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760726.224944:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545bcf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.224954:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545bcf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.224976:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545bcf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.224984:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545bcf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.225089:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04292a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495772032:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33043us (33260us total) trans 4294972730 rc 0/0 00000100:00100000:0.0:1789760726.225100:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5442 00000100:00100000:0.0:1789760726.225110:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710470846779 time 710512743721 state 0x2 ref 0 00000001:00080000:1.0:1789760726.229080:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972730 is committed 00000100:00100000:2.0:1789760726.230409:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760726.230560:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495772160 00000100:00100000:1.0:1789760726.230569:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495772160 00000100:00100000:1.0:1789760726.230624:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710470846779 time 710512743721 state 0x0 ref 1 00000100:00100000:1.0:1789760726.230640:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710513004137 time 710512743721 state 0x1 ref 1 00000100:00100000:1.0:1789760726.230704:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710513004137 time 710551421017 state 0x0 ref 1 00000100:00100000:1.0:1789760726.230717:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495772160:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760726.230775:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22175744, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:1.0:1789760726.230784:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545bd6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.230801:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760726.230806:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760726.230812:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545bd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.230825:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545bd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760726.253833:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972731 is committed 00002000:00010000:1.0:1789760726.254008:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22175744 -> 22179840 00010000:00010000:1.0:1789760726.254026:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545bd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.254038:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545bd6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.254047:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545bd6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.254053:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760726.254058:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545bd6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.254066:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545bd6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.254085:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545bd6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.254093:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545bd6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760726.254207:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495772160:12345-192.168.203.9@tcp:4:dd.0 Request processed in 23485us (23800us total) trans 4294972731 rc 0/0 00000100:00100000:1.0:1789760726.254221:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5443 00000100:00100000:1.0:1789760726.254233:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710513004137 time 710551421017 state 0x2 ref 0 00000100:00100000:2.0:1789760726.258638:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760726.259328:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495772288 00000100:00100000:0.0:1789760726.259341:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495772288 00000100:00100000:0.0:1789760726.259399:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710513004137 time 710551421017 state 0x0 ref 1 00000100:00100000:0.0:1789760726.259414:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710551681433 time 710551421017 state 0x1 ref 1 00000100:00100000:0.0:1789760726.259448:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710551681433 time 710580164557 state 0x0 ref 1 00000100:00100000:0.0:1789760726.259461:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495772288:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760726.259506:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22179840, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:0.0:1789760726.259516:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545bdd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.259536:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760726.259542:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.259548:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545bdd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.259563:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545bdd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760726.314017:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972732 is committed 00002000:00010000:0.0:1789760726.314121:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22179840 -> 22183936 00010000:00010000:0.0:1789760726.314149:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545bdd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.314169:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545bdd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.314178:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545bdd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.314185:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760726.314190:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545bdd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.314199:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545bdd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.314223:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545bdd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.314231:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545bdd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.314364:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495772288:12345-192.168.203.9@tcp:4:dd.0 Request processed in 54901us (55730us total) trans 4294972732 rc 0/0 00000100:00100000:0.0:1789760726.314376:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5444 00000100:00100000:0.0:1789760726.314385:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710551681433 time 710580164557 state 0x2 ref 0 00000100:00100000:0.0:1789760726.320250:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760726.320402:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495772416 00000100:00100000:2.0:1789760726.320412:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495772416 00000100:00100000:2.0:1789760726.320466:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710551681433 time 710580164557 state 0x0 ref 1 00000100:00100000:2.0:1789760726.320481:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710580424973 time 710580164557 state 0x1 ref 1 00000100:00100000:2.0:1789760726.320511:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710580424973 time 710641227885 state 0x0 ref 1 00000100:00100000:2.0:1789760726.320524:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495772416:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760726.320574:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22183936, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:2.0:1789760726.320583:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545be4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.320599:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760726.320603:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760726.320610:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545be4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.320623:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545be4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760726.361626:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972733 is committed 00002000:00010000:3.0:1789760726.362085:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22183936 -> 22188032 00010000:00010000:3.0:1789760726.362102:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545be4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.362114:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545be4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.362123:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545be4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.362130:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760726.362204:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545be4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.362215:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545be4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.362240:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545be4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.362249:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545be4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.362471:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495772416:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41937us (42217us total) trans 4294972733 rc 0/0 00000100:00100000:3.0:1789760726.362482:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5445 00000100:00100000:3.0:1789760726.362492:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710580424973 time 710641227885 state 0x2 ref 0 00000100:00100000:3.0:1789760726.365006:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760726.365094:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495772544 00000100:00100000:1.0:1789760726.365103:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495772544 00000100:00100000:1.0:1789760726.365166:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710580424973 time 710641227885 state 0x0 ref 1 00000100:00100000:1.0:1789760726.365182:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710641488301 time 710641227885 state 0x1 ref 1 00000100:00100000:1.0:1789760726.365210:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710641488301 time 710685927486 state 0x0 ref 1 00000100:00100000:1.0:1789760726.365223:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495772544:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760726.365268:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22188032, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:1.0:1789760726.365312:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545beb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.365328:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760726.365333:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760726.365339:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545beb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.365353:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545beb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760726.404823:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972734 is committed 00002000:00010000:1.0:1789760726.405053:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22188032 -> 22192128 00010000:00010000:1.0:1789760726.405069:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545beb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.405082:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545beb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.405091:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545beb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.405098:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760726.405103:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545beb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.405110:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545beb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.405128:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545beb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.405201:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545beb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760726.405319:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495772544:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40093us (40316us total) trans 4294972734 rc 0/0 00000100:00100000:1.0:1789760726.405329:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5446 00000100:00100000:1.0:1789760726.405338:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710641488301 time 710685927486 state 0x2 ref 0 00000100:00100000:0.0:1789760726.416108:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760726.416235:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495772672 00000100:00100000:1.0:1789760726.416244:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495772672 00000100:00100000:1.0:1789760726.416328:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710641488301 time 710685927486 state 0x0 ref 1 00000100:00100000:1.0:1789760726.416344:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710686187902 time 710685927486 state 0x1 ref 1 00000100:00100000:1.0:1789760726.416366:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710686187902 time 710737083413 state 0x0 ref 1 00000100:00100000:1.0:1789760726.416379:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495772672:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760726.416426:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22192128, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:1.0:1789760726.416435:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545bf2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.416451:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760726.416456:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760726.416469:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545bf2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.416483:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545bf2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760726.473339:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972735 is committed 00002000:00010000:3.0:1789760726.473450:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22192128 -> 22196224 00010000:00010000:3.0:1789760726.473465:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545bf2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.473486:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545bf2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.473496:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545bf2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.473503:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760726.473508:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545bf2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.473522:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545bf2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.473542:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545bf2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.473551:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545bf2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.473643:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495772672:12345-192.168.203.9@tcp:4:dd.0 Request processed in 57262us (57539us total) trans 4294972735 rc 0/0 00000100:00100000:3.0:1789760726.473654:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5447 00000100:00100000:3.0:1789760726.473664:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710686187902 time 710737083413 state 0x2 ref 0 00000100:00100000:3.0:1789760726.476964:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760726.477007:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495772800 00000100:00100000:3.0:1789760726.477017:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495772800 00000100:00100000:3.0:1789760726.477065:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710686187902 time 710737083413 state 0x0 ref 1 00000100:00100000:3.0:1789760726.477079:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710737343829 time 710737083413 state 0x1 ref 1 00000100:00100000:3.0:1789760726.477102:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710737343829 time 710797819856 state 0x0 ref 1 00000100:00100000:3.0:1789760726.477113:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495772800:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760726.477182:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22196224, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:3.0:1789760726.477191:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545bf9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.477206:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760726.477210:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.477216:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545bf9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.477230:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545bf9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760726.507892:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22196224 -> 22200320 00010000:00010000:3.0:1789760726.507911:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545bf9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.507923:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545bf9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.507932:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545bf9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.507939:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760726.507944:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545bf9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.507954:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545bf9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.507978:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545bf9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.507987:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545bf9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.508077:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495772800:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30960us (31117us total) trans 4294972736 rc 0/0 00000100:00100000:3.0:1789760726.508087:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5448 00000100:00100000:3.0:1789760726.508097:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710737343829 time 710797819856 state 0x2 ref 0 00000001:00080000:1.0:1789760726.508305:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972736 is committed 00000100:00100000:2.0:1789760726.512193:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760726.512312:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495772928 00000100:00100000:1.0:1789760726.512322:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495772928 00000100:00100000:1.0:1789760726.512372:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710737343829 time 710797819856 state 0x0 ref 1 00000100:00100000:1.0:1789760726.512385:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710798080272 time 710797819856 state 0x1 ref 1 00000100:00100000:1.0:1789760726.512412:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710798080272 time 710833128758 state 0x0 ref 1 00000100:00100000:1.0:1789760726.512422:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495772928:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760726.512464:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22200320, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:1.0:1789760726.512473:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545c00 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.512611:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760726.512616:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760726.512622:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545c00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.512636:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545c00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760726.547764:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22200320 -> 22204416 00010000:00010000:3.0:1789760726.547784:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545c00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.547797:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545c00 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.547807:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545c00 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.547814:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760726.547819:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545c00 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.547833:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545c00 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.547854:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545c00 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.547862:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545c00 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.547957:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495772928:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35530us (35767us total) trans 4294972737 rc 0/0 00000100:00100000:3.0:1789760726.547967:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5449 00000100:00100000:3.0:1789760726.547976:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710798080272 time 710833128758 state 0x2 ref 0 00000001:00080000:2.0:1789760726.548050:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972737 is committed 00000100:00100000:2.0:1789760726.555782:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760726.555873:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495773056 00000100:00100000:3.0:1789760726.555883:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495773056 00000100:00100000:3.0:1789760726.555936:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710798080272 time 710833128758 state 0x0 ref 1 00000100:00100000:3.0:1789760726.555951:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710833389174 time 710833128758 state 0x1 ref 1 00000100:00100000:3.0:1789760726.555985:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710833389174 time 710876702261 state 0x0 ref 1 00000100:00100000:3.0:1789760726.555997:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495773056:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760726.556044:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22204416, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:3.0:1789760726.556053:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545c07 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.556070:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760726.556075:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.556081:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545c07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.556094:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545c07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760726.615932:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972738 is committed 00002000:00010000:1.0:1789760726.616354:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22204416 -> 22208512 00010000:00010000:1.0:1789760726.616371:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545c07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.616389:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545c07 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.616399:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545c07 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.616406:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760726.616411:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545c07 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.616420:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545c07 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.616440:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545c07 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.616448:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545c07 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760726.616613:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495773056:12345-192.168.203.9@tcp:4:dd.0 Request processed in 60613us (60835us total) trans 4294972738 rc 0/0 00000100:00100000:1.0:1789760726.616623:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5450 00000100:00100000:1.0:1789760726.616632:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710833389174 time 710876702261 state 0x2 ref 0 00000100:00100000:1.0:1789760726.619537:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760726.619698:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495773184 00000100:00100000:3.0:1789760726.619707:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495773184 00000100:00100000:3.0:1789760726.619757:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710833389174 time 710876702261 state 0x0 ref 1 00000100:00100000:3.0:1789760726.619772:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710876962677 time 710876702261 state 0x1 ref 1 00000100:00100000:3.0:1789760726.619838:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710876962677 time 710940555192 state 0x0 ref 1 00000100:00100000:3.0:1789760726.619850:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04290700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495773184:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760726.619896:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22208512, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:3.0:1789760726.619904:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c0e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.619919:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760726.619923:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.619928:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.619942:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760726.651912:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972739 is committed 00002000:00010000:3.0:1789760726.652009:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22208512 -> 22212608 00002000:00010000:3.0:1789760726.652013:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760725 -> 1789760726 00002000:00010000:3.0:1789760726.652016:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760725 -> 1789760726 00010000:00010000:3.0:1789760726.652029:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.652042:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c0e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.652052:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c0e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.652060:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760726.652065:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c0e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.652075:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c0e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.652094:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c0e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.652107:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c0e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.652251:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04290700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495773184:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32397us (32718us total) trans 4294972739 rc 0/0 00000100:00100000:3.0:1789760726.652261:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5451 00000100:00100000:3.0:1789760726.652270:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710876962677 time 710940555192 state 0x2 ref 0 00000100:00100000:2.0:1789760726.656512:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760726.656592:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495773312 00000100:00100000:3.0:1789760726.656602:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495773312 00000100:00100000:3.0:1789760726.656652:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710876962677 time 710940555192 state 0x0 ref 1 00000100:00100000:3.0:1789760726.656668:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710940815608 time 710940555192 state 0x1 ref 1 00000100:00100000:3.0:1789760726.656703:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710940815608 time 710977419767 state 0x0 ref 1 00000100:00100000:3.0:1789760726.656716:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495773312:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760726.656762:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22212608, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:3.0:1789760726.656771:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c15 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.656787:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760726.656792:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.656798:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.656818:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760726.681509:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972740 is committed 00002000:00010000:3.0:1789760726.681631:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22212608 -> 22216704 00010000:00010000:3.0:1789760726.681647:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.681661:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c15 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.681671:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.681677:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760726.681682:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.681690:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c15 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.681710:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c15 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.681718:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c15 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.681854:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495773312:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25134us (25346us total) trans 4294972740 rc 0/0 00000100:00100000:3.0:1789760726.681865:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5452 00000100:00100000:3.0:1789760726.681875:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710940815608 time 710977419767 state 0x2 ref 0 00000100:00100000:2.0:1789760726.685666:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760726.685832:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495773440 00000100:00100000:1.0:1789760726.685842:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495773440 00000100:00100000:1.0:1789760726.685894:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710940815608 time 710977419767 state 0x0 ref 1 00000100:00100000:1.0:1789760726.685909:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710977680183 time 710977419767 state 0x1 ref 1 00000100:00100000:1.0:1789760726.685939:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710977680183 time 711006656303 state 0x0 ref 1 00000100:00100000:1.0:1789760726.685952:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495773440:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760726.685998:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22216704, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:1.0:1789760726.686007:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354545c1c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.686023:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760726.686027:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760726.686033:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354545c1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.686046:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354545c1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760726.731616:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972741 is committed 00002000:00010000:3.0:1789760726.732380:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22216704 -> 22220800 00010000:00010000:3.0:1789760726.732397:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354545c1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.732412:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354545c1c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.732422:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354545c1c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.732429:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760726.732435:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354545c1c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.732445:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354545c1c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.732466:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354545c1c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.732489:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354545c1c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.732581:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495773440:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46626us (46920us total) trans 4294972741 rc 0/0 00000100:00100000:3.0:1789760726.732590:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5453 00000100:00100000:3.0:1789760726.732599:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710977680183 time 711006656303 state 0x2 ref 0 00000100:00100000:1.0:1789760726.740020:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760726.740117:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495773568 00000100:00100000:3.0:1789760726.740128:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495773568 00000100:00100000:3.0:1789760726.740199:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 710977680183 time 711006656303 state 0x0 ref 1 00000100:00100000:3.0:1789760726.740215:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711006916719 time 711006656303 state 0x1 ref 1 00000100:00100000:3.0:1789760726.740247:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711006916719 time 711060963653 state 0x0 ref 1 00000100:00100000:3.0:1789760726.740259:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495773568:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760726.740332:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22220800, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:3.0:1789760726.740341:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c23 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.740356:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760726.740361:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.740367:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.740386:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760726.797630:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22220800 -> 22224896 00010000:00010000:3.0:1789760726.797653:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.797797:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c23 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.797807:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c23 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.797815:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760726.797820:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c23 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.797830:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c23 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.797851:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c23 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.797860:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c23 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.798124:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495773568:12345-192.168.203.9@tcp:4:dd.0 Request processed in 57859us (58108us total) trans 4294972742 rc 0/0 00000100:00100000:3.0:1789760726.799600:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5454 00000100:00100000:3.0:1789760726.799722:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711006916719 time 711060963653 state 0x2 ref 0 00000001:00080000:1.0:1789760726.801175:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972742 is committed 00000100:00100000:0.0:1789760726.816015:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760726.816219:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495773696 00000100:00100000:3.0:1789760726.816229:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495773696 00000100:00100000:3.0:1789760726.816313:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711006916719 time 711060963653 state 0x0 ref 1 00000100:00100000:3.0:1789760726.816338:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711061224069 time 711060963653 state 0x1 ref 1 00000100:00100000:3.0:1789760726.816364:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711061224069 time 711137080567 state 0x0 ref 1 00000100:00100000:3.0:1789760726.816376:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495773696:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760726.816421:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22224896, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:3.0:1789760726.816431:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c2a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.816448:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760726.816453:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.816461:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.816484:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760726.855390:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972743 is committed 00002000:00010000:2.0:1789760726.855507:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22224896 -> 22228992 00010000:00010000:2.0:1789760726.855523:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.855536:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c2a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.855545:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c2a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.855553:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760726.855558:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c2a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.855566:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c2a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.855585:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c2a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.855593:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545c2a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760726.855699:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495773696:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39321us (39690us total) trans 4294972743 rc 0/0 00000100:00100000:2.0:1789760726.855709:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5455 00000100:00100000:2.0:1789760726.855720:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711061224069 time 711137080567 state 0x2 ref 0 00000100:00100000:0.0:1789760726.861861:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760726.862720:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495773824 00000100:00100000:3.0:1789760726.862731:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495773824 00000100:00100000:3.0:1789760726.862786:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711061224069 time 711137080567 state 0x0 ref 1 00000100:00100000:3.0:1789760726.862803:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711137340983 time 711137080567 state 0x1 ref 1 00000100:00100000:3.0:1789760726.862836:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711137340983 time 711183553044 state 0x0 ref 1 00000100:00100000:3.0:1789760726.862850:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495773824:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760726.863047:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22228992, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:3.0:1789760726.863058:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c31 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.863073:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760726.863077:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.863083:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.863098:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.878548:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760726.903525:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972744 is committed 00002000:00010000:2.0:1789760726.903788:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22228992 -> 22233088 00010000:00010000:2.0:1789760726.903804:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.903817:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c31 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.903831:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c31 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.903840:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760726.903845:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c31 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.903854:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c31 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.903873:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c31 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.903881:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c31 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760726.904049:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495773824:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41197us (42194us total) trans 4294972744 rc 0/0 00000100:00100000:2.0:1789760726.904060:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5456 00000100:00100000:2.0:1789760726.904070:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711137340983 time 711183553044 state 0x2 ref 0 00000100:00100000:1.0:1789760726.911895:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760726.912214:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495773952 00000100:00100000:3.0:1789760726.912225:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495773952 00000100:00100000:3.0:1789760726.912429:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711137340983 time 711183553044 state 0x0 ref 1 00000100:00100000:3.0:1789760726.912450:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711183813460 time 711183553044 state 0x1 ref 1 00000100:00100000:3.0:1789760726.912531:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711183813460 time 711233247689 state 0x0 ref 1 00000100:00100000:3.0:1789760726.912544:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04291180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495773952:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760726.912595:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22233088, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:3.0:1789760726.912605:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c38 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.912803:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760726.912808:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.912815:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.912829:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760726.950569:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972745 is committed 00002000:00010000:1.0:1789760726.950685:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22233088 -> 22237184 00010000:00010000:1.0:1789760726.950701:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.950716:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c38 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.950724:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c38 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.950731:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760726.950735:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c38 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.950743:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c38 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.950761:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c38 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.950769:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c38 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760726.950855:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04291180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495773952:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38309us (38969us total) trans 4294972745 rc 0/0 00000100:00100000:1.0:1789760726.950865:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5457 00000100:00100000:1.0:1789760726.950875:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711183813460 time 711233247689 state 0x2 ref 0 00000100:00100000:0.0:1789760726.958881:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760726.961954:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495774080 00000100:00100000:2.0:1789760726.961972:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495774080 00000100:00100000:2.0:1789760726.962048:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711183813460 time 711233247689 state 0x0 ref 1 00000100:00100000:2.0:1789760726.962064:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711233508105 time 711233247689 state 0x1 ref 1 00000100:00100000:2.0:1789760726.962089:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711233508105 time 711282806209 state 0x0 ref 1 00000100:00100000:2.0:1789760726.962102:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495774080:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760726.962174:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22237184, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:2.0:1789760726.962183:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545c3f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.962200:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760726.962205:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760726.962211:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545c3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.962227:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545c3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760727.006900:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22237184 -> 22241280 00010000:00010000:0.0:1789760727.006920:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545c3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.006934:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545c3f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.006942:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545c3f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.006950:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760727.006955:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545c3f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.006963:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545c3f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.006983:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545c3f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.006991:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545c3f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.007088:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495774080:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44984us (48215us total) trans 4294972746 rc 0/0 00000100:00100000:0.0:1789760727.007099:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5458 00000100:00100000:0.0:1789760727.007109:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711233508105 time 711282806209 state 0x2 ref 0 00000001:00080000:2.0:1789760727.008450:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972746 is committed 00000100:00100000:0.0:1789760727.013029:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760727.013144:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495774208 00000100:00100000:2.0:1789760727.013170:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495774208 00000100:00100000:2.0:1789760727.013248:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711233508105 time 711282806209 state 0x0 ref 1 00000100:00100000:2.0:1789760727.013263:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711283066625 time 711282806209 state 0x1 ref 1 00000100:00100000:2.0:1789760727.013335:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711283066625 time 711334050956 state 0x0 ref 1 00000100:00100000:2.0:1789760727.013349:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495774208:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760727.013405:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22241280, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:2.0:1789760727.013416:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545c46 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.013433:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760727.013438:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760727.013445:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545c46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.013460:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545c46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760727.013868:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760727.077744:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972747 is committed 00002000:00010000:3.0:1789760727.077848:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22241280 -> 22245376 00010000:00010000:3.0:1789760727.077868:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545c46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.077879:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545c46 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.077886:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545c46 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.077896:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760727.077902:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545c46 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.077910:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545c46 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.077935:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545c46 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.077943:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545c46 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.078097:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495774208:12345-192.168.203.9@tcp:4:dd.0 Request processed in 64745us (65075us total) trans 4294972747 rc 0/0 00000100:00100000:3.0:1789760727.078110:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5459 00000100:00100000:3.0:1789760727.078124:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711283066625 time 711334050956 state 0x2 ref 0 00000100:00100000:1.0:1789760727.085939:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760727.086117:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495774336 00000100:00100000:3.0:1789760727.086139:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495774336 00000100:00100000:3.0:1789760727.086413:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711283066625 time 711334050956 state 0x0 ref 1 00000100:00100000:3.0:1789760727.086433:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711334311372 time 711334050956 state 0x1 ref 1 00000100:00100000:3.0:1789760727.086470:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711334311372 time 711407186801 state 0x0 ref 1 00000100:00100000:3.0:1789760727.086490:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495774336:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760727.086837:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22245376, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:3.0:1789760727.086849:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c4d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.086871:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760727.086876:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.086883:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.086900:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760727.164643:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972748 is committed 00002000:00010000:3.0:1789760727.164939:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22245376 -> 22249472 00010000:00010000:3.0:1789760727.164961:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.164978:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c4d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.164989:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c4d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.164996:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760727.165001:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c4d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.165012:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c4d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.165037:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c4d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.165046:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c4d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.165171:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495774336:12345-192.168.203.9@tcp:4:dd.0 Request processed in 78678us (79243us total) trans 4294972748 rc 0/0 00000100:00100000:3.0:1789760727.165184:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5460 00000100:00100000:3.0:1789760727.165197:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711334311372 time 711407186801 state 0x2 ref 0 00000100:00100000:0.0:1789760727.182361:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760727.182491:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495774464 00000100:00100000:3.0:1789760727.182503:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495774464 00000100:00100000:3.0:1789760727.182675:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711334311372 time 711407186801 state 0x0 ref 1 00000100:00100000:3.0:1789760727.182696:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711407447217 time 711407186801 state 0x1 ref 1 00000100:00100000:3.0:1789760727.182796:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711407447217 time 711503511317 state 0x0 ref 1 00000100:00100000:3.0:1789760727.182811:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495774464:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760727.182877:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22249472, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:3.0:1789760727.182898:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c54 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.183106:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760727.183111:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.183119:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.183137:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760727.233822:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972749 is committed 00002000:00010000:0.0:1789760727.233941:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22249472 -> 22253568 00010000:00010000:0.0:1789760727.233994:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.234014:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c54 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.234022:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c54 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.234028:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760727.234033:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c54 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.234043:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c54 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.234066:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c54 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.234072:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545c54 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.234464:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495774464:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51647us (52185us total) trans 4294972749 rc 0/0 00000100:00100000:0.0:1789760727.234477:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5461 00000100:00100000:0.0:1789760727.234490:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711407447217 time 711503511317 state 0x2 ref 0 00000100:00100000:0.0:1789760727.238682:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760727.241473:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495774592 00000100:00100000:3.0:1789760727.241498:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495774592 00000100:00100000:3.0:1789760727.241641:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711407447217 time 711503511317 state 0x0 ref 1 00000100:00100000:3.0:1789760727.241668:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711503771733 time 711503511317 state 0x1 ref 1 00000100:00100000:3.0:1789760727.241714:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711503771733 time 711562429806 state 0x0 ref 1 00000100:00100000:3.0:1789760727.241733:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495774592:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760727.241809:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22253568, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:3.0:1789760727.241819:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c5b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.241841:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760727.241847:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.241855:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.241890:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760727.289251:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972750 is committed 00002000:00010000:3.0:1789760727.289455:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22253568 -> 22257664 00010000:00010000:3.0:1789760727.289476:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.289491:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c5b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.289502:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c5b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.289509:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760727.289527:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c5b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.289537:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c5b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.289565:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c5b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.289573:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c5b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.289699:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495774592:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47965us (51025us total) trans 4294972750 rc 0/0 00000100:00100000:3.0:1789760727.289833:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5462 00000100:00100000:3.0:1789760727.289847:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711503771733 time 711562429806 state 0x2 ref 0 00000100:00100000:1.0:1789760727.295622:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760727.295956:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495774720 00000100:00100000:3.0:1789760727.295973:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495774720 00000100:00100000:3.0:1789760727.296050:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711503771733 time 711562429806 state 0x0 ref 1 00000100:00100000:3.0:1789760727.296135:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711562690222 time 711562429806 state 0x1 ref 1 00000100:00100000:3.0:1789760727.296181:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711562690222 time 711616897524 state 0x0 ref 1 00000100:00100000:3.0:1789760727.296197:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495774720:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760727.296264:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22257664, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:3.0:1789760727.296274:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c62 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.296309:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760727.296313:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.296319:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.296331:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.327310:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff929036f88a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520304256:0@lo:400:kworker.0 00000100:00100000:0.0:1789760727.327400:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760727.327473:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789760727.327480:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff929036f8a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520304384:0@lo:400:kworker.0 00000100:00100000:0.0:1789760727.327507:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760727.327615:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520304256 00000100:00100000:0.0:1789760727.327624:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520304256 00000100:00100000:0.0:1789760727.327945:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 467 00000100:00100000:0.0:1789760727.327956:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fdcb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520304256:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760727.328227:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fdcb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520304256:12345-0@lo:400:kworker.0 Request processed in 271us (825us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760727.328237:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 467 00000100:00100000:0.0:1789760727.328268:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520304384 00000100:00100000:0.0:1789760727.328273:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520304384 00000100:00100000:0.0:1789760727.328332:0:6709:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 218 00000100:00100000:0.0:1789760727.328339:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fdc9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520304384:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760727.328372:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fdc9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520304384:12345-0@lo:400:kworker.0 Request processed in 33us (865us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760727.328379:0:6709:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 218 00000100:00080000:0.0:1789760727.328396:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789760727.328400:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760727.328431:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff929036f8a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520304384:0@lo:400:kworker.0 00000100:00100000:0.0:1789760727.328443:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff929036f88a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520304256:0@lo:400:kworker.0 00000100:00100000:1.0:1789760727.329267:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff929036f88380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:59c4e264-9b75-482b-8813-3d1dc124f384:3317:1876699520304000:0@lo:400:kworker.0 00000100:00100000:1.0:1789760727.329370:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760727.329415:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3317:1876699520304512:0@lo:400:kworker.0 00000100:00100000:1.0:1789760727.329443:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760727.329471:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1789760727.329478:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff929036f8a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3317:1876699520304128:0@lo:400:kworker.0 00000100:00100000:1.0:1789760727.329499:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760727.329531:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3317:1876699520304640:0@lo:400:kworker.0 00000100:00100000:1.0:1789760727.329553:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760727.329586:0:12710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520304512 00000100:00100000:1.0:1789760727.329593:0:12710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520304512 00000100:00100000:1.0:1789760727.329644:0:12710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520304128 00000100:00100000:1.0:1789760727.329647:0:12710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520304128 00000100:00100000:1.0:1789760727.329667:0:12710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 468 00000100:00100000:1.0:1789760727.329675:0:12710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1876699520304512:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760727.329950:0:12710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1876699520304512:12345-0@lo:400:kworker.0 Request processed in 274us (504us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760727.329958:0:12710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 468 00000100:00100000:1.0:1789760727.329979:0:12710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 469 00000100:00100000:1.0:1789760727.329986:0:12710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1876699520304128:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760727.330018:0:12710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1876699520304128:12345-0@lo:400:kworker.0 Request processed in 33us (517us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760727.330025:0:12710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 469 00000100:00100000:1.0:1789760727.330070:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff929036f8a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3317:1876699520304128:0@lo:400:kworker.0 00000100:00100000:1.0:1789760727.330083:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3317:1876699520304512:0@lo:400:kworker.0 00000100:00100000:0.0:1789760727.330369:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520304000 00000100:00100000:0.0:1789760727.330375:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520304000 00000100:00100000:0.0:1789760727.330407:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 395 00000100:00100000:0.0:1789760727.330415:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929012317450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:59c4e264-9b75-482b-8813-3d1dc124f384+19:3317:x1876699520304000:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760727.330463:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929012317450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:59c4e264-9b75-482b-8813-3d1dc124f384+19:3317:x1876699520304000:12345-0@lo:400:kworker.0 Request processed in 49us (1093us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760727.330471:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 395 00000100:00080000:1.0:1789760727.330972:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760727.331123:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff929036f88380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:59c4e264-9b75-482b-8813-3d1dc124f384:3317:1876699520304000:0@lo:400:kworker.0 00000100:00100000:3.0:1789760727.331446:0:10490:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520304640 00000100:00100000:3.0:1789760727.331453:0:10490:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520304640 00000100:00100000:3.0:1789760727.331502:0:10490:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 219 00000100:00100000:3.0:1789760727.331511:0:10490:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3317:x1876699520304640:12345-0@lo:400:kworker.0 00000100:00080000:0.0:1789760727.331520:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789760727.331634:0:10490:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3317:x1876699520304640:12345-0@lo:400:kworker.0 Request processed in 124us (2079us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760727.331642:0:10490:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 219 00000100:00100000:0.0:1789760727.331701:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3317:1876699520304640:0@lo:400:kworker.0 00000001:00080000:3.0:1789760727.358669:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972751 is committed 00002000:00010000:1.0:1789760727.359118:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22257664 -> 22261760 00010000:00010000:1.0:1789760727.359159:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.359178:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c62 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.359189:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c62 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.359197:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760727.359203:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c62 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.359215:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c62 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.359240:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c62 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.359248:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545c62 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.359569:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495774720:12345-192.168.203.9@tcp:4:dd.0 Request processed in 63363us (63950us total) trans 4294972751 rc 0/0 00000100:00100000:0.0:1789760727.359584:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5463 00000100:00100000:0.0:1789760727.359599:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711562690222 time 711616897524 state 0x2 ref 0 00000100:00100000:1.0:1789760727.365783:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760727.365910:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495774848 00000100:00100000:0.0:1789760727.365925:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495774848 00000100:00100000:0.0:1789760727.366001:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711562690222 time 711616897524 state 0x0 ref 1 00000100:00100000:0.0:1789760727.366020:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711617157940 time 711616897524 state 0x1 ref 1 00000100:00100000:0.0:1789760727.366059:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711617157940 time 711686775288 state 0x0 ref 1 00000100:00100000:0.0:1789760727.366073:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495774848:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760727.366154:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22261760, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:0.0:1789760727.366165:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545c69 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.366185:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760727.366190:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760727.366197:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545c69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.366211:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545c69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760727.464494:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22261760 -> 22265856 00010000:00010000:3.0:1789760727.464524:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545c69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.464539:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545c69 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.464549:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545c69 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.464559:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760727.464565:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545c69 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.464574:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545c69 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.464604:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545c69 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.464613:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545c69 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.464737:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495774848:12345-192.168.203.9@tcp:4:dd.0 Request processed in 98659us (98959us total) trans 4294972752 rc 0/0 00000100:00100000:3.0:1789760727.464750:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5464 00000100:00100000:3.0:1789760727.464763:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711617157940 time 711686775288 state 0x2 ref 0 00000001:00080000:3.0:1789760727.464821:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972752 is committed 00000100:00100000:3.0:1789760727.469639:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760727.469864:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495774976 00000100:00100000:0.0:1789760727.469878:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495774976 00000100:00100000:0.0:1789760727.469954:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711617157940 time 711686775288 state 0x0 ref 1 00000100:00100000:0.0:1789760727.469975:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711687035704 time 711686775288 state 0x1 ref 1 00000100:00100000:0.0:1789760727.470011:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711687035704 time 711790727396 state 0x0 ref 1 00000100:00100000:0.0:1789760727.470026:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495774976:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760727.470092:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22265856, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:0.0:1789760727.470104:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545c70 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.470123:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760727.470140:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760727.470157:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545c70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.470174:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545c70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760727.513915:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972753 is committed 00002000:00010000:0.0:1789760727.514095:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22265856 -> 22269952 00010000:00010000:0.0:1789760727.514116:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545c70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.514154:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545c70 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.514166:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545c70 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.514175:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760727.514243:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545c70 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.514255:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545c70 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.514323:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545c70 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.514332:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545c70 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.514520:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495774976:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44489us (44890us total) trans 4294972753 rc 0/0 00000100:00100000:0.0:1789760727.514537:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5465 00000100:00100000:0.0:1789760727.514550:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711687035704 time 711790727396 state 0x2 ref 0 00000100:00100000:1.0:1789760727.517989:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760727.520064:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495775104 00000100:00100000:0.0:1789760727.520087:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495775104 00000100:00100000:0.0:1789760727.520326:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711687035704 time 711790727396 state 0x0 ref 1 00000100:00100000:0.0:1789760727.520348:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711790987812 time 711790727396 state 0x1 ref 1 00000100:00100000:0.0:1789760727.520657:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711790987812 time 711841370850 state 0x0 ref 1 00000100:00100000:0.0:1789760727.520676:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495775104:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760727.520746:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22269952, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:0.0:1789760727.520758:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545c77 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.520778:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760727.520783:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760727.520791:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545c77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.520807:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545c77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760727.574953:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972754 is committed 00002000:00010000:0.0:1789760727.575109:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22269952 -> 22274048 00010000:00010000:0.0:1789760727.575152:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545c77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.575169:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545c77 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.575246:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545c77 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.575257:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760727.575263:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545c77 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.575274:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545c77 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.575321:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545c77 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.575331:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545c77 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.576034:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495775104:12345-192.168.203.9@tcp:4:dd.0 Request processed in 55355us (58052us total) trans 4294972754 rc 0/0 00000100:00100000:0.0:1789760727.576049:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5466 00000100:00100000:0.0:1789760727.576063:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711790987812 time 711841370850 state 0x2 ref 0 00000100:00100000:2.0:1789760727.582344:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760727.582428:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495775232 00000100:00100000:3.0:1789760727.582441:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495775232 00000100:00100000:3.0:1789760727.582510:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711790987812 time 711841370850 state 0x0 ref 1 00000100:00100000:3.0:1789760727.582528:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711841631266 time 711841370850 state 0x1 ref 1 00000100:00100000:3.0:1789760727.582562:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711841631266 time 711903279501 state 0x0 ref 1 00000100:00100000:3.0:1789760727.582575:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f921c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495775232:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760727.582636:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22274048, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:3.0:1789760727.582647:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545c7e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.582667:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760727.582673:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.582681:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545c7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.582699:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545c7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760727.628633:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22274048 -> 22278144 00010000:00010000:1.0:1789760727.628661:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545c7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.628676:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545c7e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.628685:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545c7e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.628692:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760727.628697:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545c7e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.628706:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545c7e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.628729:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545c7e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.628736:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545c7e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760727.628865:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f921c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495775232:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46285us (46531us total) trans 4294972755 rc 0/0 00000100:00100000:1.0:1789760727.628878:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5467 00000100:00100000:1.0:1789760727.628891:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711841631266 time 711903279501 state 0x2 ref 0 00000001:00080000:3.0:1789760727.629035:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972755 is committed 00000100:00100000:3.0:1789760727.635128:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760727.635560:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495775360 00000100:00100000:0.0:1789760727.635575:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495775360 00000100:00100000:0.0:1789760727.635651:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711841631266 time 711903279501 state 0x0 ref 1 00000100:00100000:0.0:1789760727.635671:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711903539917 time 711903279501 state 0x1 ref 1 00000100:00100000:0.0:1789760727.635708:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711903539917 time 711956424168 state 0x0 ref 1 00000100:00100000:0.0:1789760727.635722:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495775360:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760727.635787:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22278144, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:0.0:1789760727.635799:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545c85 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.635820:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760727.635826:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760727.635834:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545c85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.635852:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545c85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760727.682229:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22278144 -> 22282240 00002000:00010000:1.0:1789760727.682241:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760726 -> 1789760727 00002000:00010000:1.0:1789760727.682244:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760726 -> 1789760727 00010000:00010000:1.0:1789760727.682261:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545c85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.682302:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545c85 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.682312:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545c85 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.682320:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760727.682326:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545c85 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.682335:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545c85 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.682361:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545c85 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.682369:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545c85 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760727.682509:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495775360:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46781us (47396us total) trans 4294972756 rc 0/0 00000100:00100000:1.0:1789760727.682521:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5468 00000100:00100000:1.0:1789760727.682533:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711903539917 time 711956424168 state 0x2 ref 0 00000001:00080000:2.0:1789760727.685353:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972756 is committed 00000100:00100000:2.0:1789760727.686775:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760727.686877:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495775488 00000100:00100000:0.0:1789760727.686890:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495775488 00000100:00100000:0.0:1789760727.686964:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711903539917 time 711956424168 state 0x0 ref 1 00000100:00100000:0.0:1789760727.686983:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711956684584 time 711956424168 state 0x1 ref 1 00000100:00100000:0.0:1789760727.687019:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711956684584 time 712007735281 state 0x0 ref 1 00000100:00100000:0.0:1789760727.687037:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f923100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495775488:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760727.687101:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22282240, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:0.0:1789760727.687112:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545c8c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.687151:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760727.687157:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760727.687165:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545c8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.687189:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545c8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760727.748910:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972757 is committed 00002000:00010000:1.0:1789760727.750801:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22282240 -> 22286336 00010000:00010000:1.0:1789760727.750826:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545c8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.750841:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545c8c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.750852:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545c8c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.750860:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760727.750866:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545c8c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.750876:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545c8c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.750901:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545c8c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.750915:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545c8c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760727.751084:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f923100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495775488:12345-192.168.203.9@tcp:4:dd.0 Request processed in 64043us (64313us total) trans 4294972757 rc 0/0 00000100:00100000:1.0:1789760727.751097:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5469 00000100:00100000:1.0:1789760727.751110:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711956684584 time 712007735281 state 0x2 ref 0 00000100:00100000:2.0:1789760727.757663:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760727.757905:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495775616 00000100:00100000:1.0:1789760727.757919:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495775616 00000100:00100000:1.0:1789760727.757995:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 711956684584 time 712007735281 state 0x0 ref 1 00000100:00100000:1.0:1789760727.758015:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712007995697 time 712007735281 state 0x1 ref 1 00000100:00100000:1.0:1789760727.758098:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712007995697 time 712078813733 state 0x0 ref 1 00000100:00100000:1.0:1789760727.758112:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f922300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495775616:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760727.758198:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22286336, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:1.0:1789760727.758209:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545c93 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.758227:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760727.758232:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760727.758238:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545c93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.758260:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545c93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760727.802621:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972758 is committed 00002000:00010000:0.0:1789760727.802729:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22286336 -> 22290432 00010000:00010000:0.0:1789760727.802751:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545c93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.802766:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545c93 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.802776:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545c93 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.802784:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760727.802789:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545c93 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.802800:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545c93 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.802825:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545c93 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.802833:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545c93 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.802996:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f922300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495775616:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44878us (45340us total) trans 4294972758 rc 0/0 00000100:00100000:0.0:1789760727.803007:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5470 00000100:00100000:0.0:1789760727.803021:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712007995697 time 712078813733 state 0x2 ref 0 00000100:00100000:3.0:1789760727.806605:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760727.806727:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495775744 00000100:00100000:1.0:1789760727.806740:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495775744 00000100:00100000:1.0:1789760727.806820:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712007995697 time 712078813733 state 0x0 ref 1 00000100:00100000:1.0:1789760727.806838:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712079074149 time 712078813733 state 0x1 ref 1 00000100:00100000:1.0:1789760727.806875:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712079074149 time 712127591894 state 0x0 ref 1 00000100:00100000:1.0:1789760727.806889:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495775744:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760727.806956:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22290432, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:1.0:1789760727.806968:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545c9a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.806989:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760727.806994:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760727.807002:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545c9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.807018:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545c9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.841606:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f02769c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520304768:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760727.841705:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760727.842307:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520304768 00000100:00100000:2.0:1789760727.842321:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520304768 00000100:00100000:2.0:1789760727.842392:0:6711:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 194 00000100:00100000:2.0:1789760727.842403:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520304768:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760727.842450:0:6711:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760727.842530:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520304768:12345-0@lo:13:osp-pre-1-0.0 Request processed in 129us (825us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760727.842539:0:6711:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 194 00000100:00100000:0.0:1789760727.842831:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f02769c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520304768:0@lo:13:osp-pre-1-0.0 00002000:00010000:1.0:1789760727.859250:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22290432 -> 22294528 00010000:00010000:1.0:1789760727.859308:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545c9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.859325:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545c9a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.859334:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545c9a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.859342:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760727.859348:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545c9a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.859358:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545c9a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.859754:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545c9a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.859765:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545c9a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760727.859889:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495775744:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52996us (53293us total) trans 4294972759 rc 0/0 00000100:00100000:1.0:1789760727.859902:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5471 00000100:00100000:1.0:1789760727.859914:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712079074149 time 712127591894 state 0x2 ref 0 00000001:00080000:3.0:1789760727.860033:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972759 is committed 00000100:00100000:2.0:1789760727.864033:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760727.864158:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495775872 00000100:00100000:1.0:1789760727.864173:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495775872 00000100:00100000:1.0:1789760727.864254:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712079074149 time 712127591894 state 0x0 ref 1 00000100:00100000:1.0:1789760727.864273:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712127852310 time 712127591894 state 0x1 ref 1 00000100:00100000:1.0:1789760727.864356:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712127852310 time 712185072250 state 0x0 ref 1 00000100:00100000:1.0:1789760727.864380:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495775872:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760727.864449:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22294528, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:1.0:1789760727.864459:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545ca1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.864750:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760727.864756:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760727.864763:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545ca1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.864781:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545ca1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760727.900244:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972760 is committed 00002000:00010000:2.0:1789760727.900430:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22294528 -> 22298624 00010000:00010000:2.0:1789760727.900453:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545ca1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.900600:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545ca1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.900611:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545ca1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.900618:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760727.900624:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545ca1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.900634:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545ca1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.900658:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545ca1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.900667:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545ca1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760727.900797:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495775872:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36422us (36772us total) trans 4294972760 rc 0/0 00000100:00100000:2.0:1789760727.900809:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5472 00000100:00100000:2.0:1789760727.900822:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712127852310 time 712185072250 state 0x2 ref 0 00010000:00010000:1.0:1789760727.903988:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789760727.906007:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760727.906133:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495776000 00000100:00100000:0.0:1789760727.906156:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495776000 00000100:00100000:0.0:1789760727.906243:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712127852310 time 712185072250 state 0x0 ref 1 00000100:00100000:0.0:1789760727.906262:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712185332666 time 712185072250 state 0x1 ref 1 00000100:00100000:0.0:1789760727.906325:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712185332666 time 712227040272 state 0x0 ref 1 00000100:00100000:0.0:1789760727.906344:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495776000:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760727.906427:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22298624, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:0.0:1789760727.906438:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545ca8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.906459:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760727.906464:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760727.906482:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545ca8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.906500:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545ca8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760727.941257:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22298624 -> 22302720 00010000:00010000:1.0:1789760727.941307:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545ca8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.941332:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545ca8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.941342:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545ca8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.941352:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760727.941358:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545ca8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.941368:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545ca8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.941527:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545ca8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.941537:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545ca8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760727.941723:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972761 is committed 00000100:00100000:1.0:1789760727.941793:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495776000:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35445us (35796us total) trans 4294972761 rc 0/0 00000100:00100000:1.0:1789760727.941806:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5473 00000100:00100000:1.0:1789760727.941820:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712185332666 time 712227040272 state 0x2 ref 0 00000100:00100000:3.0:1789760727.945884:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760727.945961:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495776128 00000100:00100000:1.0:1789760727.945975:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495776128 00000100:00100000:1.0:1789760727.946051:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712185332666 time 712227040272 state 0x0 ref 1 00000100:00100000:1.0:1789760727.946070:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712227300688 time 712227040272 state 0x1 ref 1 00000100:00100000:1.0:1789760727.946111:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712227300688 time 712266826437 state 0x0 ref 1 00000100:00100000:1.0:1789760727.946134:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495776128:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760727.946213:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22302720, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:1.0:1789760727.946225:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545caf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.946246:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760727.946252:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760727.946261:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545caf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.946310:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545caf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760727.979903:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972762 is committed 00002000:00010000:2.0:1789760727.980043:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22302720 -> 22306816 00010000:00010000:2.0:1789760727.980066:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545caf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.980082:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545caf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.980093:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545caf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.980102:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760727.980108:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545caf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.980200:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545caf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.980228:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545caf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.980236:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545caf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.980671:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495776128:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34540us (34793us total) trans 4294972762 rc 0/0 00000100:00100000:3.0:1789760727.980685:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5474 00000100:00100000:3.0:1789760727.980698:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712227300688 time 712266826437 state 0x2 ref 0 00000100:00100000:0.0:1789760727.984078:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760727.984256:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495776256 00000100:00100000:2.0:1789760727.984270:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495776256 00000100:00100000:2.0:1789760727.984540:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712227300688 time 712266826437 state 0x0 ref 1 00000100:00100000:2.0:1789760727.984559:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712267086853 time 712266826437 state 0x1 ref 1 00000100:00100000:2.0:1789760727.984645:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712267086853 time 712305360435 state 0x0 ref 1 00000100:00100000:2.0:1789760727.984661:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495776256:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760727.984727:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22306816, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:2.0:1789760727.984739:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545cb6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.984760:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760727.984766:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760727.984775:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545cb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.984792:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545cb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760728.023568:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972763 is committed 00002000:00010000:2.0:1789760728.023790:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22306816 -> 22310912 00010000:00010000:2.0:1789760728.023810:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545cb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.023825:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545cb6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.023839:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545cb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.023847:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760728.023853:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545cb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.023862:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545cb6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.023887:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545cb6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.023895:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545cb6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760728.024057:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495776256:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39391us (39987us total) trans 4294972763 rc 0/0 00000100:00100000:2.0:1789760728.024070:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5475 00000100:00100000:2.0:1789760728.024082:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712267086853 time 712305360435 state 0x2 ref 0 00000100:00100000:3.0:1789760728.027710:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760728.027788:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495776384 00000100:00100000:1.0:1789760728.027801:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495776384 00000100:00100000:1.0:1789760728.027868:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712267086853 time 712305360435 state 0x0 ref 1 00000100:00100000:1.0:1789760728.027885:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712305620851 time 712305360435 state 0x1 ref 1 00000100:00100000:1.0:1789760728.027970:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712305620851 time 712348684483 state 0x0 ref 1 00000100:00100000:1.0:1789760728.027988:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495776384:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760728.028050:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22310912, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:1.0:1789760728.028061:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545cbd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.028081:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760728.028086:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.028093:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545cbd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.028110:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545cbd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760728.076848:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22310912 -> 22315008 00010000:00010000:0.0:1789760728.076878:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545cbd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.076895:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545cbd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.076906:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545cbd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.076920:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760728.076926:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545cbd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.076935:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545cbd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.076961:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545cbd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.076970:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545cbd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.077130:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495776384:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49129us (49417us total) trans 4294972764 rc 0/0 00000100:00100000:0.0:1789760728.077239:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5476 00000100:00100000:0.0:1789760728.077253:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712305620851 time 712348684483 state 0x2 ref 0 00000001:00080000:2.0:1789760728.080363:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972764 is committed 00000100:00100000:2.0:1789760728.081258:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760728.081438:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495776512 00000100:00100000:3.0:1789760728.081451:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495776512 00000100:00100000:3.0:1789760728.081600:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712305620851 time 712348684483 state 0x0 ref 1 00000100:00100000:3.0:1789760728.081622:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712348944899 time 712348684483 state 0x1 ref 1 00000100:00100000:3.0:1789760728.081699:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712348944899 time 712402414458 state 0x0 ref 1 00000100:00100000:3.0:1789760728.081714:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495776512:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760728.081781:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22315008, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:3.0:1789760728.081791:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545cc4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.081812:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760728.081817:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760728.081826:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545cc4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.081842:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545cc4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760728.119633:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972765 is committed 00002000:00010000:3.0:1789760728.119713:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22315008 -> 22319104 00010000:00010000:3.0:1789760728.119732:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545cc4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.119749:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545cc4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.119759:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545cc4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.119773:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760728.119780:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545cc4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.119791:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545cc4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.119815:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545cc4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.119824:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545cc4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760728.119943:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495776512:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38225us (38931us total) trans 4294972765 rc 0/0 00000100:00100000:3.0:1789760728.119957:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5477 00000100:00100000:3.0:1789760728.119970:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712348944899 time 712402414458 state 0x2 ref 0 00000100:00100000:1.0:1789760728.122956:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760728.123034:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495776640 00000100:00100000:2.0:1789760728.123048:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495776640 00000100:00100000:2.0:1789760728.123132:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712348944899 time 712402414458 state 0x0 ref 1 00000100:00100000:2.0:1789760728.123160:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712402674874 time 712402414458 state 0x1 ref 1 00000100:00100000:2.0:1789760728.123195:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712402674874 time 712443911578 state 0x0 ref 1 00000100:00100000:2.0:1789760728.123209:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495776640:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760728.123275:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22319104, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:2.0:1789760728.123320:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545ccb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.123340:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760728.123345:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760728.123353:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545ccb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.123369:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545ccb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760728.175555:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972766 is committed 00002000:00010000:2.0:1789760728.175729:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22319104 -> 22323200 00010000:00010000:2.0:1789760728.175750:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545ccb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.175766:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545ccb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.175782:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545ccb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.175791:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760728.175798:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545ccb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.175808:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545ccb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.175834:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545ccb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.175843:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545ccb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760728.176008:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495776640:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52794us (53063us total) trans 4294972766 rc 0/0 00000100:00100000:2.0:1789760728.176021:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5478 00000100:00100000:2.0:1789760728.176033:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712402674874 time 712443911578 state 0x2 ref 0 00000100:00100000:2.0:1789760728.179698:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760728.179818:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495776768 00000100:00100000:0.0:1789760728.179831:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495776768 00000100:00100000:0.0:1789760728.179905:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712402674874 time 712443911578 state 0x0 ref 1 00000100:00100000:0.0:1789760728.179925:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712444171994 time 712443911578 state 0x1 ref 1 00000100:00100000:0.0:1789760728.179962:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712444171994 time 712500678329 state 0x0 ref 1 00000100:00100000:0.0:1789760728.179977:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495776768:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760728.180043:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22323200, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:0.0:1789760728.180054:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545cd2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.180075:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760728.180079:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.180086:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545cd2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.180102:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545cd2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760728.228559:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972767 is committed 00002000:00010000:1.0:1789760728.228784:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22323200 -> 22327296 00010000:00010000:1.0:1789760728.228807:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545cd2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.228823:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545cd2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.228835:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545cd2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.228844:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760728.228851:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545cd2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.228862:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545cd2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.228890:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545cd2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.228899:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545cd2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.229072:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495776768:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49089us (49382us total) trans 4294972767 rc 0/0 00000100:00100000:1.0:1789760728.229086:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5479 00000100:00100000:1.0:1789760728.229100:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712444171994 time 712500678329 state 0x2 ref 0 00000100:00100000:1.0:1789760728.233473:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760728.233772:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495776896 00000100:00100000:3.0:1789760728.233785:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495776896 00000100:00100000:3.0:1789760728.233858:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712444171994 time 712500678329 state 0x0 ref 1 00000100:00100000:3.0:1789760728.233878:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712500938745 time 712500678329 state 0x1 ref 1 00000100:00100000:3.0:1789760728.233912:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712500938745 time 712554628579 state 0x0 ref 1 00000100:00100000:3.0:1789760728.233931:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495776896:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760728.233998:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22327296, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:3.0:1789760728.234009:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545cd9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.234030:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760728.234035:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760728.234043:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545cd9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.234060:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545cd9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760728.269506:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972768 is committed 00002000:00010000:2.0:1789760728.269651:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22327296 -> 22331392 00010000:00010000:2.0:1789760728.269673:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545cd9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.269690:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545cd9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.269700:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545cd9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.269708:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760728.269714:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545cd9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.269724:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545cd9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.269751:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545cd9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.269760:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545cd9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760728.269885:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495776896:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35950us (36427us total) trans 4294972768 rc 0/0 00000100:00100000:2.0:1789760728.269898:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5480 00000100:00100000:2.0:1789760728.269911:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712500938745 time 712554628579 state 0x2 ref 0 00000100:00100000:1.0:1789760728.275485:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760728.275575:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495777024 00000100:00100000:2.0:1789760728.275598:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495777024 00000100:00100000:2.0:1789760728.275670:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712500938745 time 712554628579 state 0x0 ref 1 00000100:00100000:2.0:1789760728.275688:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712554888995 time 712554628579 state 0x1 ref 1 00000100:00100000:2.0:1789760728.275729:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712554888995 time 712596444996 state 0x0 ref 1 00000100:00100000:2.0:1789760728.275743:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495777024:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760728.275807:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22331392, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:2.0:1789760728.275823:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545ce0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.275845:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760728.275850:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760728.275857:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545ce0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.275873:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545ce0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760728.306875:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22331392 -> 22335488 00010000:00010000:2.0:1789760728.306904:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545ce0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.306922:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545ce0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.306938:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545ce0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.306947:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760728.306953:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545ce0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.306963:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545ce0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.306987:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545ce0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.306995:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545ce0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760728.307126:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495777024:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31369us (31641us total) trans 4294972769 rc 0/0 00000100:00100000:2.0:1789760728.307146:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5481 00000100:00100000:2.0:1789760728.307159:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712554888995 time 712596444996 state 0x2 ref 0 00000001:00080000:0.0:1789760728.309018:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972769 is committed 00000100:00100000:0.0:1789760728.311414:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760728.311654:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495777152 00000100:00100000:1.0:1789760728.311667:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495777152 00000100:00100000:1.0:1789760728.311744:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712554888995 time 712596444996 state 0x0 ref 1 00000100:00100000:1.0:1789760728.311764:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712596705412 time 712596444996 state 0x1 ref 1 00000100:00100000:1.0:1789760728.311798:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712596705412 time 712632514115 state 0x0 ref 1 00000100:00100000:1.0:1789760728.311812:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495777152:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760728.311879:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22335488, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:1.0:1789760728.311890:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545ce7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.311910:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760728.311915:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.311922:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545ce7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.311939:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545ce7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760728.348728:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972770 is committed 00002000:00010000:1.0:1789760728.348971:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22335488 -> 22339584 00010000:00010000:1.0:1789760728.348996:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545ce7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.349013:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545ce7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.349023:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545ce7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.349031:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760728.349037:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545ce7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.349046:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545ce7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.349073:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545ce7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.349081:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354545ce7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.349226:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495777152:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37409us (37818us total) trans 4294972770 rc 0/0 00000100:00100000:1.0:1789760728.349238:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5482 00000100:00100000:1.0:1789760728.349251:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712596705412 time 712632514115 state 0x2 ref 0 00000100:00100000:1.0:1789760728.353466:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760728.355418:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495777280 00000100:00100000:2.0:1789760728.355439:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495777280 00000100:00100000:2.0:1789760728.355536:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712596705412 time 712632514115 state 0x0 ref 1 00000100:00100000:2.0:1789760728.355557:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712632774531 time 712632514115 state 0x1 ref 1 00000100:00100000:2.0:1789760728.355596:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712632774531 time 712676312110 state 0x0 ref 1 00000100:00100000:2.0:1789760728.355611:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495777280:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760728.355677:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22339584, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:2.0:1789760728.355688:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545cee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.355710:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760728.355716:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760728.355724:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545cee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.355742:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545cee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760728.439844:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972771 is committed 00002000:00010000:1.0:1789760728.440106:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22339584 -> 22343680 00010000:00010000:1.0:1789760728.440145:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545cee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.440163:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545cee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.440172:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545cee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.440181:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760728.440186:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545cee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.440201:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545cee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.440225:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545cee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.440234:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354545cee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.440406:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495777280:12345-192.168.203.9@tcp:4:dd.0 Request processed in 84790us (86946us total) trans 4294972771 rc 0/0 00000100:00100000:1.0:1789760728.440420:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5483 00000100:00100000:1.0:1789760728.440434:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712632774531 time 712676312110 state 0x2 ref 0 00000100:00100000:0.0:1789760728.444022:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760728.444207:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495777408 00000100:00100000:2.0:1789760728.444221:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495777408 00000100:00100000:2.0:1789760728.444554:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712632774531 time 712676312110 state 0x0 ref 1 00000100:00100000:2.0:1789760728.444574:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712676572526 time 712676312110 state 0x1 ref 1 00000100:00100000:2.0:1789760728.444638:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712676572526 time 712765353850 state 0x0 ref 1 00000100:00100000:2.0:1789760728.444653:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495777408:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760728.444719:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22343680, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:2.0:1789760728.444731:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545cf5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.444750:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760728.444755:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760728.444763:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545cf5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.444779:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545cf5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760728.488329:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972772 is committed 00002000:00010000:2.0:1789760728.488575:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22343680 -> 22347776 00010000:00010000:2.0:1789760728.488596:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545cf5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.488612:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545cf5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.488622:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545cf5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.488630:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760728.488636:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545cf5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.488647:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545cf5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.488680:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545cf5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.488688:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545cf5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760728.488851:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495777408:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44194us (44836us total) trans 4294972772 rc 0/0 00000100:00100000:2.0:1789760728.488865:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5484 00000100:00100000:2.0:1789760728.488877:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712676572526 time 712765353850 state 0x2 ref 0 00000100:00100000:2.0:1789760728.492103:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760728.492258:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495777536 00000100:00100000:1.0:1789760728.492271:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495777536 00000100:00100000:1.0:1789760728.492408:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712676572526 time 712765353850 state 0x0 ref 1 00000100:00100000:1.0:1789760728.492428:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712765614266 time 712765353850 state 0x1 ref 1 00000100:00100000:1.0:1789760728.492561:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712765614266 time 712813276552 state 0x0 ref 1 00000100:00100000:1.0:1789760728.492576:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495777536:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760728.492801:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22347776, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:1.0:1789760728.492813:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354545cfc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.492835:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760728.492841:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.492850:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354545cfc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.492866:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354545cfc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760728.529002:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972773 is committed 00002000:00010000:2.0:1789760728.529042:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22347776 -> 22351872 00010000:00010000:2.0:1789760728.529062:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354545cfc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.529077:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354545cfc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.529086:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354545cfc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.529095:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760728.529101:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354545cfc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.529127:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354545cfc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.529162:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354545cfc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.529184:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354545cfc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760728.529360:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495777536:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36779us (37265us total) trans 4294972773 rc 0/0 00000100:00100000:2.0:1789760728.529374:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5485 00000100:00100000:2.0:1789760728.529388:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712765614266 time 712813276552 state 0x2 ref 0 00000100:00100000:1.0:1789760728.533593:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760728.536356:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495777664 00000100:00100000:2.0:1789760728.536378:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495777664 00000100:00100000:2.0:1789760728.536541:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712765614266 time 712813276552 state 0x0 ref 1 00000100:00100000:2.0:1789760728.536563:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712813536968 time 712813276552 state 0x1 ref 1 00000100:00100000:2.0:1789760728.536659:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712813536968 time 712857374087 state 0x0 ref 1 00000100:00100000:2.0:1789760728.536675:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495777664:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760728.536743:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22351872, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:2.0:1789760728.536754:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d03 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.536774:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760728.536779:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760728.536787:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.536803:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760728.561446:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972774 is committed 00002000:00010000:2.0:1789760728.563231:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22351872 -> 22355968 00010000:00010000:2.0:1789760728.563257:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.563273:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d03 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.563307:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.563316:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760728.563322:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.563332:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d03 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.563359:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d03 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.563368:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d03 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760728.563510:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495777664:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26830us (29923us total) trans 4294972774 rc 0/0 00000100:00100000:2.0:1789760728.563523:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5486 00000100:00100000:2.0:1789760728.563536:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712813536968 time 712857374087 state 0x2 ref 0 00000100:00100000:3.0:1789760728.563735:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760728.564248:0:12710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495777920 00000100:00100000:2.0:1789760728.564260:0:12710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495777920 00000100:00100000:2.0:1789760728.564534:0:12710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 470 00000100:00100000:2.0:1789760728.564546:0:12710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:17022965-1e4f-46fb-915c-61730096b8e2+8:2356:x1876699495777920:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:2.0:1789760728.564624:0:12710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:17022965-1e4f-46fb-915c-61730096b8e2+8:2356:x1876699495777920:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 79us (889us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760728.564631:0:12710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 470 00000100:00080000:0.0:1789760728.565019:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760728.567235:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760728.567307:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760728.567360:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495778048 00000100:00100000:0.0:1789760728.567368:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760728.567373:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495778048 00000100:00080000:0.0:1789760728.567414:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760728.567439:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 471 00000100:00100000:3.0:1789760728.567450:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd02a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2357:x1876699495778048:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:3.0:1789760728.567545:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd02a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2357:x1876699495778048:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 95us (236us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760728.567724:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495777792 00000100:00100000:2.0:1789760728.567729:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495777792 00000100:00100000:2.0:1789760728.567767:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 396 00000100:00100000:2.0:1789760728.567774:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929012313850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2357:x1876699495777792:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:2.0:1789760728.567806:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929012313850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2357:x1876699495777792:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 33us (586us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760728.567812:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 396 00000100:00100000:2.0:1789760728.567855:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495778176 00000100:00100000:2.0:1789760728.567858:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495778176 00000100:00100000:2.0:1789760728.567902:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712813536968 time 712857374087 state 0x0 ref 1 00000100:00100000:2.0:1789760728.567919:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712857634503 time 712857374087 state 0x1 ref 1 00000100:00100000:2.0:1789760728.567942:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712857634503 time 712888658476 state 0x0 ref 1 00000100:00100000:2.0:1789760728.567952:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495778176:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760728.568007:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22355968, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:2.0:1789760728.568019:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545d0a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.568039:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760728.568044:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760728.568052:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545d0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.568069:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545d0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760728.575268:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 471 00000001:00080000:3.0:1789760728.604519:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972775 is committed 00002000:00010000:3.0:1789760728.604974:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22355968 -> 22360064 00010000:00010000:3.0:1789760728.604995:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545d0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.605010:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545d0a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.605020:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545d0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.605028:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760728.605040:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545d0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.605050:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545d0a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.605077:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545d0a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.605086:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545d0a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760728.605374:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495778176:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37414us (37996us total) trans 4294972775 rc 0/0 00000100:00100000:3.0:1789760728.605388:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5487 00000100:00100000:3.0:1789760728.605401:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712857634503 time 712888658476 state 0x2 ref 0 00000100:00100000:1.0:1789760728.608253:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760728.608366:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495778304 00000100:00100000:2.0:1789760728.608379:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495778304 00000100:00100000:2.0:1789760728.608455:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712857634503 time 712888658476 state 0x0 ref 1 00000100:00100000:2.0:1789760728.608487:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712888918892 time 712888658476 state 0x1 ref 1 00000100:00100000:2.0:1789760728.608647:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712888918892 time 712929362251 state 0x0 ref 1 00000100:00100000:2.0:1789760728.608663:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495778304:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760728.608732:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22360064, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:2.0:1789760728.608742:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d11 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.608762:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760728.608767:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760728.608774:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.608864:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760728.668357:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972776 is committed 00002000:00010000:2.0:1789760728.668617:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22360064 -> 22364160 00010000:00010000:2.0:1789760728.668639:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.668654:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d11 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.668664:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d11 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.668672:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760728.668677:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d11 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.668689:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d11 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.668715:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d11 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.668724:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d11 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760728.668856:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495778304:12345-192.168.203.9@tcp:4:dd.0 Request processed in 60189us (60610us total) trans 4294972776 rc 0/0 00000100:00100000:2.0:1789760728.668869:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5488 00000100:00100000:2.0:1789760728.668882:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712888918892 time 712929362251 state 0x2 ref 0 00000100:00100000:1.0:1789760728.679583:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760728.679691:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495778432 00000100:00100000:2.0:1789760728.679704:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495778432 00000100:00100000:2.0:1789760728.679774:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712888918892 time 712929362251 state 0x0 ref 1 00000100:00100000:2.0:1789760728.679807:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712929622667 time 712929362251 state 0x1 ref 1 00000100:00100000:2.0:1789760728.679840:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712929622667 time 713000556675 state 0x0 ref 1 00000100:00100000:2.0:1789760728.679853:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495778432:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760728.679919:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22364160, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:2.0:1789760728.679929:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d18 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.679949:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760728.679955:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760728.679962:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.679978:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760728.713307:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972777 is committed 00002000:00010000:3.0:1789760728.713396:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22364160 -> 22368256 00002000:00010000:3.0:1789760728.713401:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760727 -> 1789760728 00002000:00010000:3.0:1789760728.713404:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760727 -> 1789760728 00010000:00010000:3.0:1789760728.713422:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.713438:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d18 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.713448:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.713456:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760728.713462:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.713473:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d18 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.713504:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d18 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.713513:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545d18 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760728.713645:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495778432:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33787us (34071us total) trans 4294972777 rc 0/0 00000100:00100000:3.0:1789760728.713659:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5489 00000100:00100000:3.0:1789760728.713672:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712929622667 time 713000556675 state 0x2 ref 0 00000100:00100000:1.0:1789760728.717589:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760728.717697:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495778560 00000100:00100000:3.0:1789760728.717711:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495778560 00000100:00100000:3.0:1789760728.717788:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 712929622667 time 713000556675 state 0x0 ref 1 00000100:00100000:3.0:1789760728.717809:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713000817091 time 713000556675 state 0x1 ref 1 00000100:00100000:3.0:1789760728.717901:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713000817091 time 713038616193 state 0x0 ref 1 00000100:00100000:3.0:1789760728.717916:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495778560:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760728.717990:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22368256, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:3.0:1789760728.718001:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d1f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.718023:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760728.718028:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760728.718036:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.718052:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760728.753184:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972778 is committed 00002000:00010000:3.0:1789760728.753762:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22368256 -> 22372352 00010000:00010000:3.0:1789760728.753784:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.753800:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d1f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.753811:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d1f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.753820:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760728.753826:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d1f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.753836:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d1f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.753866:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d1f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.753874:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d1f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760728.754054:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495778560:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36134us (36476us total) trans 4294972778 rc 0/0 00000100:00100000:3.0:1789760728.754070:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5490 00000100:00100000:3.0:1789760728.754083:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713000817091 time 713038616193 state 0x2 ref 0 00000100:00100000:1.0:1789760728.761032:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760728.765751:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495778688 00000100:00100000:2.0:1789760728.765776:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495778688 00000100:00100000:2.0:1789760728.765971:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713000817091 time 713038616193 state 0x0 ref 1 00000100:00100000:2.0:1789760728.765992:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713038876609 time 713038616193 state 0x1 ref 1 00000100:00100000:2.0:1789760728.766078:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713038876609 time 713086792933 state 0x0 ref 1 00000100:00100000:2.0:1789760728.766096:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495778688:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760728.766187:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22372352, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:2.0:1789760728.766199:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545d26 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.766220:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760728.766224:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760728.766232:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545d26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.766249:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545d26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760728.835100:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972779 is committed 00002000:00010000:2.0:1789760728.835307:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22372352 -> 22376448 00010000:00010000:2.0:1789760728.835329:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545d26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.835345:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545d26 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.835354:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545d26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.835362:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760728.835367:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545d26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.835376:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545d26 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.835404:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545d26 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.835412:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545d26 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760728.835730:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495778688:12345-192.168.203.9@tcp:4:dd.0 Request processed in 69631us (74709us total) trans 4294972779 rc 0/0 00000100:00100000:2.0:1789760728.835748:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5491 00000100:00100000:2.0:1789760728.835762:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713038876609 time 713086792933 state 0x2 ref 0 00000100:00100000:1.0:1789760728.842247:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760728.842479:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495778816 00000100:00100000:3.0:1789760728.842492:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495778816 00000100:00100000:3.0:1789760728.842565:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713038876609 time 713086792933 state 0x0 ref 1 00000100:00100000:3.0:1789760728.842584:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713087053349 time 713086792933 state 0x1 ref 1 00000100:00100000:3.0:1789760728.842651:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713087053349 time 713163367102 state 0x0 ref 1 00000100:00100000:3.0:1789760728.842666:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495778816:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760728.842735:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22376448, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:3.0:1789760728.842746:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d2d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.842769:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760728.842774:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760728.842856:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.842939:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760728.903405:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972780 is committed 00002000:00010000:3.0:1789760728.903661:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22376448 -> 22380544 00010000:00010000:3.0:1789760728.903684:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.903701:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d2d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.903711:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.903720:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760728.903725:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.903735:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.903760:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d2d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.903768:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d2d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760728.904083:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495778816:12345-192.168.203.9@tcp:4:dd.0 Request processed in 61411us (61843us total) trans 4294972780 rc 0/0 00000100:00100000:3.0:1789760728.904097:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5492 00000100:00100000:3.0:1789760728.904110:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713087053349 time 713163367102 state 0x2 ref 0 00000100:00100000:1.0:1789760728.913172:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760728.913354:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495778944 00000100:00100000:3.0:1789760728.913368:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495778944 00000100:00100000:3.0:1789760728.913446:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713087053349 time 713163367102 state 0x0 ref 1 00000100:00100000:3.0:1789760728.913474:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713163627518 time 713163367102 state 0x1 ref 1 00000100:00100000:3.0:1789760728.913507:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713163627518 time 713234223210 state 0x0 ref 1 00000100:00100000:3.0:1789760728.913522:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495778944:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760728.913600:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22380544, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:3.0:1789760728.913611:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545d34 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.913632:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760728.913637:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760728.913644:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545d34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.913661:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545d34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.927659:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760728.946880:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22380544 -> 22384640 00010000:00010000:3.0:1789760728.946909:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545d34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.946924:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545d34 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.946940:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545d34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.946949:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760728.946956:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545d34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.946967:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545d34 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.946993:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545d34 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.947002:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545d34 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760728.947143:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495778944:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33607us (33973us total) trans 4294972781 rc 0/0 00000100:00100000:3.0:1789760728.947156:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5493 00000100:00100000:3.0:1789760728.947171:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713163627518 time 713234223210 state 0x2 ref 0 00000001:00080000:1.0:1789760728.947526:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972781 is committed 00000100:00100000:3.0:1789760728.951510:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760728.951867:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495779072 00000100:00100000:1.0:1789760728.951882:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495779072 00000100:00100000:1.0:1789760728.951969:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713163627518 time 713234223210 state 0x0 ref 1 00000100:00100000:1.0:1789760728.951989:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713234483626 time 713234223210 state 0x1 ref 1 00000100:00100000:1.0:1789760728.952030:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713234483626 time 713272746381 state 0x0 ref 1 00000100:00100000:1.0:1789760728.952049:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495779072:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760728.952126:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22384640, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:1.0:1789760728.952147:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545d3b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.952169:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760728.952174:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.952181:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545d3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.952198:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545d3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760728.995412:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22384640 -> 22388736 00010000:00010000:0.0:1789760728.995549:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545d3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760728.995556:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972782 is committed 00010000:00010000:0.0:1789760728.995572:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545d3b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.995581:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545d3b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.995590:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760728.995596:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545d3b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.995608:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545d3b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.995632:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545d3b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.995639:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354545d3b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.995793:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495779072:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43739us (44291us total) trans 4294972782 rc 0/0 00000100:00100000:0.0:1789760728.995807:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5494 00000100:00100000:0.0:1789760728.995821:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713234483626 time 713272746381 state 0x2 ref 0 00000100:00100000:0.0:1789760728.999962:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760729.000083:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495779200 00000100:00100000:3.0:1789760729.000098:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495779200 00000100:00100000:3.0:1789760729.000382:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713234483626 time 713272746381 state 0x0 ref 1 00000100:00100000:3.0:1789760729.000406:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713273006797 time 713272746381 state 0x1 ref 1 00000100:00100000:3.0:1789760729.000446:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713273006797 time 713321162531 state 0x0 ref 1 00000100:00100000:3.0:1789760729.001024:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495779200:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760729.001097:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22388736, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:3.0:1789760729.001109:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545d42 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.001157:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760729.001163:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.001172:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545d42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.001188:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545d42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760729.002090:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789760729.047993:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22388736 -> 22392832 00010000:00010000:2.0:1789760729.048024:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545d42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.048042:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545d42 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.048052:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545d42 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.048061:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760729.048067:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545d42 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.048076:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545d42 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.048101:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545d42 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.048110:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545d42 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.048421:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495779200:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47422us (49307us total) trans 4294972783 rc 0/0 00000100:00100000:2.0:1789760729.048806:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5495 00000100:00100000:2.0:1789760729.048820:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713273006797 time 713321162531 state 0x2 ref 0 00000001:00080000:0.0:1789760729.049641:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972783 is committed 00000100:00100000:1.0:1789760729.059998:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760729.063700:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495779328 00000100:00100000:3.0:1789760729.063722:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495779328 00000100:00100000:3.0:1789760729.063814:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713273006797 time 713321162531 state 0x0 ref 1 00000100:00100000:3.0:1789760729.063835:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713321422947 time 713321162531 state 0x1 ref 1 00000100:00100000:3.0:1789760729.063879:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713321422947 time 713384594328 state 0x0 ref 1 00000100:00100000:3.0:1789760729.063894:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495779328:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760729.063965:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22392832, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:3.0:1789760729.063977:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545d49 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.063999:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760729.064004:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.064012:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545d49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.064029:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545d49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760729.111241:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22392832 -> 22396928 00010000:00010000:3.0:1789760729.111271:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545d49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.111310:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545d49 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.111321:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545d49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.111329:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760729.111335:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545d49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.111346:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545d49 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.111375:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545d49 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.111385:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545d49 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.111634:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495779328:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47736us (51644us total) trans 4294972784 rc 0/0 00000100:00100000:3.0:1789760729.111648:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5496 00000100:00100000:3.0:1789760729.111663:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713321422947 time 713384594328 state 0x2 ref 0 00000001:00080000:2.0:1789760729.111972:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972784 is committed 00000100:00100000:3.0:1789760729.114067:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760729.114323:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495779456 00000100:00100000:0.0:1789760729.114337:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495779456 00000100:00100000:0.0:1789760729.114412:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713321422947 time 713384594328 state 0x0 ref 1 00000100:00100000:0.0:1789760729.114432:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713384854744 time 713384594328 state 0x1 ref 1 00000100:00100000:0.0:1789760729.114575:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713384854744 time 713435289948 state 0x0 ref 1 00000100:00100000:0.0:1789760729.114591:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903639f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495779456:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760729.114657:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22396928, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:0.0:1789760729.114669:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545d50 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.114690:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760729.114696:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.114705:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545d50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.114722:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545d50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760729.159882:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972785 is committed 00002000:00010000:0.0:1789760729.160006:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22396928 -> 22401024 00010000:00010000:0.0:1789760729.160026:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545d50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.160041:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545d50 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.160052:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545d50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.160061:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760729.160067:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545d50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.160077:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545d50 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.160100:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545d50 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.160108:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545d50 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.160569:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903639f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495779456:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45859us (46395us total) trans 4294972785 rc 0/0 00000100:00100000:0.0:1789760729.160583:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5497 00000100:00100000:0.0:1789760729.160596:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713384854744 time 713435289948 state 0x2 ref 0 00000100:00100000:0.0:1789760729.163820:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760729.163953:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495779584 00000100:00100000:2.0:1789760729.163967:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495779584 00000100:00100000:2.0:1789760729.164049:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713384854744 time 713435289948 state 0x0 ref 1 00000100:00100000:2.0:1789760729.164069:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713435550364 time 713435289948 state 0x1 ref 1 00000100:00100000:2.0:1789760729.164327:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713435550364 time 713485041732 state 0x0 ref 1 00000100:00100000:2.0:1789760729.164345:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495779584:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760729.164414:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22401024, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:2.0:1789760729.164426:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545d57 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.164448:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760729.164640:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.164649:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545d57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.164673:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545d57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.204270:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972786 is committed 00002000:00010000:0.0:1789760729.204520:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22401024 -> 22405120 00010000:00010000:0.0:1789760729.204538:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545d57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.204557:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545d57 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.204568:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545d57 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.204576:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760729.204581:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545d57 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.204591:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545d57 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.204615:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545d57 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.204624:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545d57 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.204744:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495779584:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40396us (40933us total) trans 4294972786 rc 0/0 00000100:00100000:0.0:1789760729.204757:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5498 00000100:00100000:0.0:1789760729.204770:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713435550364 time 713485041732 state 0x2 ref 0 00000100:00100000:1.0:1789760729.208064:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760729.208394:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495779712 00000100:00100000:3.0:1789760729.208408:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495779712 00000100:00100000:3.0:1789760729.208589:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713435550364 time 713485041732 state 0x0 ref 1 00000100:00100000:3.0:1789760729.208608:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713485302148 time 713485041732 state 0x1 ref 1 00000100:00100000:3.0:1789760729.208679:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713485302148 time 713529394600 state 0x0 ref 1 00000100:00100000:3.0:1789760729.208694:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03095500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495779712:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760729.208757:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22405120, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:3.0:1789760729.208768:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545d5e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.208792:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760729.208797:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.208806:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545d5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.208822:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545d5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760729.240477:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22405120 -> 22409216 00010000:00010000:0.0:1789760729.240502:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545d5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.240519:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545d5e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.240529:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545d5e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.240537:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760729.240543:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545d5e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.240553:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545d5e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.240575:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545d5e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.240583:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545d5e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.240694:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495779712:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31997us (32639us total) trans 4294972787 rc 0/0 00000100:00100000:0.0:1789760729.240706:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5499 00000100:00100000:0.0:1789760729.240718:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713485302148 time 713529394600 state 0x2 ref 0 00000100:00100000:0.0:1789760729.244255:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760729.244417:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495779840 00000100:00100000:3.0:1789760729.244431:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495779840 00000100:00100000:3.0:1789760729.244519:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713485302148 time 713529394600 state 0x0 ref 1 00000100:00100000:3.0:1789760729.244538:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713529655016 time 713529394600 state 0x1 ref 1 00000100:00100000:3.0:1789760729.244572:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713529655016 time 713565288921 state 0x0 ref 1 00000100:00100000:3.0:1789760729.244586:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495779840:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760729.244650:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22409216, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:3.0:1789760729.244661:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d65 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.244680:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760729.244686:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.244694:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.244714:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.251255:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972787 is committed 00000001:00080000:3.0:1789760729.284877:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972788 is committed 00002000:00010000:0.0:1789760729.285162:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22409216 -> 22413312 00010000:00010000:0.0:1789760729.285183:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.285200:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d65 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.285210:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d65 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.285220:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760729.285226:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d65 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.285236:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d65 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.285262:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d65 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.285271:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545d65 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.285417:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495779840:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40826us (41170us total) trans 4294972788 rc 0/0 00000100:00100000:0.0:1789760729.285433:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5500 00000100:00100000:0.0:1789760729.285445:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713529655016 time 713565288921 state 0x2 ref 0 00000100:00100000:3.0:1789760729.291923:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760729.292058:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495779968 00000100:00100000:0.0:1789760729.292074:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495779968 00000100:00100000:0.0:1789760729.292173:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713529655016 time 713565288921 state 0x0 ref 1 00000100:00100000:0.0:1789760729.292193:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713565549337 time 713565288921 state 0x1 ref 1 00000100:00100000:0.0:1789760729.292232:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713565549337 time 713612947898 state 0x0 ref 1 00000100:00100000:0.0:1789760729.292247:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495779968:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760729.292350:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22413312, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:0.0:1789760729.292361:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545d6c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.292381:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760729.292387:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.292394:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545d6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.292411:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545d6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.328474:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972789 is committed 00002000:00010000:0.0:1789760729.328720:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22413312 -> 22417408 00010000:00010000:0.0:1789760729.328740:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545d6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.328755:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545d6c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.328764:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545d6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.328771:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760729.328777:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545d6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.328786:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545d6c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.328812:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545d6c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.328821:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545d6c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.328953:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495779968:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36701us (37048us total) trans 4294972789 rc 0/0 00000100:00100000:0.0:1789760729.328966:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5501 00000100:00100000:0.0:1789760729.328978:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713565549337 time 713612947898 state 0x2 ref 0 00000100:00100000:0.0:1789760729.332679:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760729.332758:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495780096 00000100:00100000:3.0:1789760729.332876:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495780096 00000100:00100000:3.0:1789760729.332951:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713565549337 time 713612947898 state 0x0 ref 1 00000100:00100000:3.0:1789760729.332971:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713613208314 time 713612947898 state 0x1 ref 1 00000100:00100000:3.0:1789760729.333008:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713613208314 time 713653724010 state 0x0 ref 1 00000100:00100000:3.0:1789760729.333022:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495780096:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760729.333084:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22417408, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:3.0:1789760729.333096:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d73 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.333126:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760729.333140:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.333214:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.333232:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760729.398785:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972790 is committed 00002000:00010000:1.0:1789760729.399561:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22417408 -> 22421504 00010000:00010000:1.0:1789760729.399584:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.399601:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d73 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.399618:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d73 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.399626:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760729.399632:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d73 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.399643:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d73 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.399669:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d73 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.399678:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d73 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.399952:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495780096:12345-192.168.203.9@tcp:4:dd.0 Request processed in 66925us (67278us total) trans 4294972790 rc 0/0 00000100:00100000:1.0:1789760729.399966:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5502 00000100:00100000:1.0:1789760729.399979:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713613208314 time 713653724010 state 0x2 ref 0 00000100:00100000:1.0:1789760729.409662:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760729.409796:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495780224 00000100:00100000:0.0:1789760729.409809:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495780224 00000100:00100000:0.0:1789760729.409881:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713613208314 time 713653724010 state 0x0 ref 1 00000100:00100000:0.0:1789760729.409900:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713653984426 time 713653724010 state 0x1 ref 1 00000100:00100000:0.0:1789760729.409936:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713653984426 time 713730651864 state 0x0 ref 1 00000100:00100000:0.0:1789760729.409950:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03097b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495780224:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760729.410021:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22421504, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:0.0:1789760729.410033:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545d7a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.410053:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760729.410059:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.410067:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545d7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.410084:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545d7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.443469:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972791 is committed 00002000:00010000:3.0:1789760729.443660:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22421504 -> 22425600 00010000:00010000:3.0:1789760729.443679:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545d7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.443692:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545d7a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.443702:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545d7a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.443710:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760729.443715:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545d7a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.443726:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545d7a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.443751:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545d7a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.443760:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354545d7a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.443887:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03097b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495780224:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33932us (34232us total) trans 4294972791 rc 0/0 00000100:00100000:3.0:1789760729.443902:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5503 00000100:00100000:3.0:1789760729.443916:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713653984426 time 713730651864 state 0x2 ref 0 00000100:00100000:3.0:1789760729.455960:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760729.456088:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495780352 00000100:00100000:0.0:1789760729.456101:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495780352 00000100:00100000:0.0:1789760729.456189:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713653984426 time 713730651864 state 0x0 ref 1 00000100:00100000:0.0:1789760729.456209:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713730912280 time 713730651864 state 0x1 ref 1 00000100:00100000:0.0:1789760729.456350:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713730912280 time 713777064928 state 0x0 ref 1 00000100:00100000:0.0:1789760729.456368:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495780352:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760729.456438:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22425600, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:0.0:1789760729.456450:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545d81 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.456482:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760729.456489:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.456497:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545d81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.456516:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545d81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.486004:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972792 is committed 00002000:00010000:3.0:1789760729.486196:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22425600 -> 22429696 00010000:00010000:3.0:1789760729.486216:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545d81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.486232:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545d81 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.486243:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545d81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.486251:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760729.486257:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545d81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.486268:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545d81 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.486321:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545d81 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.486329:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545d81 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.487558:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495780352:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30084us (30504us total) trans 4294972792 rc 0/0 00000100:00100000:3.0:1789760729.487578:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5504 00000100:00100000:3.0:1789760729.487592:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713730912280 time 713777064928 state 0x2 ref 0 00000100:00100000:1.0:1789760729.489710:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760729.489849:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495780480 00000100:00100000:3.0:1789760729.489863:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495780480 00000100:00100000:3.0:1789760729.489935:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713730912280 time 713777064928 state 0x0 ref 1 00000100:00100000:3.0:1789760729.489955:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713777325344 time 713777064928 state 0x1 ref 1 00000100:00100000:3.0:1789760729.489993:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713777325344 time 713810709877 state 0x0 ref 1 00000100:00100000:3.0:1789760729.490008:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03094700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495780480:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760729.490074:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22429696, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:3.0:1789760729.490086:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545d88 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.490107:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760729.490224:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.490234:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545d88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.490253:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545d88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.521212:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972793 is committed 00002000:00010000:3.0:1789760729.521379:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22429696 -> 22433792 00010000:00010000:3.0:1789760729.521401:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545d88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.521417:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545d88 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.521428:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545d88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.521436:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760729.521443:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545d88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.521465:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545d88 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.521493:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545d88 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.521501:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545d88 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.521620:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03094700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495780480:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31607us (31916us total) trans 4294972793 rc 0/0 00000100:00100000:3.0:1789760729.521636:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5505 00000100:00100000:3.0:1789760729.521651:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713777325344 time 713810709877 state 0x2 ref 0 00000100:00100000:1.0:1789760729.530330:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760729.530515:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495780608 00000100:00100000:3.0:1789760729.530528:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495780608 00000100:00100000:3.0:1789760729.530604:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713777325344 time 713810709877 state 0x0 ref 1 00000100:00100000:3.0:1789760729.530624:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713810970293 time 713810709877 state 0x1 ref 1 00000100:00100000:3.0:1789760729.530659:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713810970293 time 713851375772 state 0x0 ref 1 00000100:00100000:3.0:1789760729.530673:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03095180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495780608:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760729.530745:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22433792, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:3.0:1789760729.530757:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545d8f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.530777:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760729.530782:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.530789:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545d8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.530807:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545d8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.567954:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972794 is committed 00002000:00010000:2.0:1789760729.568267:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22433792 -> 22437888 00010000:00010000:2.0:1789760729.568374:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545d8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.568390:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545d8f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.568401:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545d8f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.568409:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760729.568415:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545d8f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.568425:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545d8f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.568451:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545d8f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.568459:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545d8f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.568693:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495780608:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38013us (38380us total) trans 4294972794 rc 0/0 00000100:00100000:3.0:1789760729.568710:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5506 00000100:00100000:3.0:1789760729.568723:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713810970293 time 713851375772 state 0x2 ref 0 00000100:00100000:1.0:1789760729.572265:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760729.572459:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495780736 00000100:00100000:3.0:1789760729.572471:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495780736 00000100:00100000:3.0:1789760729.572541:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713810970293 time 713851375772 state 0x0 ref 1 00000100:00100000:3.0:1789760729.572561:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713851636188 time 713851375772 state 0x1 ref 1 00000100:00100000:3.0:1789760729.572598:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713851636188 time 713893313240 state 0x0 ref 1 00000100:00100000:3.0:1789760729.572615:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495780736:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760729.572688:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22437888, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:3.0:1789760729.572701:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d96 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.572722:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760729.572728:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.572736:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.572754:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.609488:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972795 is committed 00002000:00010000:3.0:1789760729.609635:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22437888 -> 22441984 00010000:00010000:3.0:1789760729.609656:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.609674:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d96 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.609686:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d96 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.609695:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760729.609701:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d96 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.609710:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d96 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.609737:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d96 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.609747:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d96 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.609955:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495780736:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37336us (37696us total) trans 4294972795 rc 0/0 00000100:00100000:3.0:1789760729.609969:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5507 00000100:00100000:3.0:1789760729.609983:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713851636188 time 713893313240 state 0x2 ref 0 00000100:00100000:1.0:1789760729.619226:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760729.619355:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495780864 00000100:00100000:3.0:1789760729.619369:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495780864 00000100:00100000:3.0:1789760729.619461:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713851636188 time 713893313240 state 0x0 ref 1 00000100:00100000:3.0:1789760729.619482:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713893573656 time 713893313240 state 0x1 ref 1 00000100:00100000:3.0:1789760729.619522:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713893573656 time 713940237559 state 0x0 ref 1 00000100:00100000:3.0:1789760729.619537:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495780864:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760729.619608:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22441984, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:3.0:1789760729.619883:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d9d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.619906:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760729.619911:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.619919:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.619936:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760729.656726:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22441984 -> 22446080 00002000:00010000:0.0:1789760729.656739:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760728 -> 1789760729 00002000:00010000:0.0:1789760729.656742:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760728 -> 1789760729 00010000:00010000:0.0:1789760729.656759:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.656775:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d9d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.656785:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.656793:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760729.656800:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.656809:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d9d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.656833:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d9d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.656840:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545d9d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.657002:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495780864:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37459us (37783us total) trans 4294972796 rc 0/0 00000100:00100000:0.0:1789760729.657016:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5508 00000100:00100000:0.0:1789760729.657028:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713893573656 time 713940237559 state 0x2 ref 0 00000001:00080000:2.0:1789760729.657145:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972796 is committed 00000100:00100000:1.0:1789760729.661687:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760729.661794:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495780992 00000100:00100000:3.0:1789760729.661807:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495780992 00000100:00100000:3.0:1789760729.661885:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713893573656 time 713940237559 state 0x0 ref 1 00000100:00100000:3.0:1789760729.661904:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713940497975 time 713940237559 state 0x1 ref 1 00000100:00100000:3.0:1789760729.661974:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713940497975 time 713982689820 state 0x0 ref 1 00000100:00100000:3.0:1789760729.661993:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495780992:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760729.662059:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22446080, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:3.0:1789760729.662070:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545da4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.662091:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760729.662096:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.662105:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545da4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.662137:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545da4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760729.696034:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972797 is committed 00002000:00010000:3.0:1789760729.696909:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22446080 -> 22450176 00010000:00010000:3.0:1789760729.696931:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545da4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.696945:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545da4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.696955:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545da4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.696963:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760729.696970:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545da4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.696980:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545da4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.697009:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545da4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.697017:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545da4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.697165:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495780992:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35166us (35484us total) trans 4294972797 rc 0/0 00000100:00100000:3.0:1789760729.697177:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5509 00000100:00100000:3.0:1789760729.697190:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713940497975 time 713982689820 state 0x2 ref 0 00000100:00100000:1.0:1789760729.700791:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760729.701362:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495781120 00000100:00100000:3.0:1789760729.701375:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495781120 00000100:00100000:3.0:1789760729.701448:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713940497975 time 713982689820 state 0x0 ref 1 00000100:00100000:3.0:1789760729.701479:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713982950236 time 713982689820 state 0x1 ref 1 00000100:00100000:3.0:1789760729.701512:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713982950236 time 714022228990 state 0x0 ref 1 00000100:00100000:3.0:1789760729.701526:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495781120:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760729.701593:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22450176, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:3.0:1789760729.701604:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545dab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.701626:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760729.701631:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.701639:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545dab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.701655:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545dab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760729.740759:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22450176 -> 22454272 00010000:00010000:0.0:1789760729.740788:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545dab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.740805:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545dab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.740815:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545dab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.740829:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760729.740835:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545dab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.740846:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545dab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.740871:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545dab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.740879:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545dab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.741002:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495781120:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39471us (40220us total) trans 4294972798 rc 0/0 00000100:00100000:0.0:1789760729.741015:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5510 00000100:00100000:0.0:1789760729.741029:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713982950236 time 714022228990 state 0x2 ref 0 00000001:00080000:2.0:1789760729.742862:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972798 is committed 00000100:00100000:1.0:1789760729.744869:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760729.744989:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495781248 00000100:00100000:0.0:1789760729.745004:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495781248 00000100:00100000:0.0:1789760729.745081:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 713982950236 time 714022228990 state 0x0 ref 1 00000100:00100000:0.0:1789760729.745101:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714022489406 time 714022228990 state 0x1 ref 1 00000100:00100000:0.0:1789760729.745249:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714022489406 time 714065963278 state 0x0 ref 1 00000100:00100000:0.0:1789760729.745268:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495781248:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760729.745365:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22454272, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:0.0:1789760729.745376:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545db2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.745398:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760729.745403:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.745411:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545db2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.745427:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545db2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760729.780264:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972799 is committed 00002000:00010000:1.0:1789760729.780475:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22454272 -> 22458368 00010000:00010000:1.0:1789760729.780497:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545db2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.780514:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545db2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.780525:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545db2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.780534:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760729.780540:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545db2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.780551:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545db2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.780579:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545db2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.780595:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545db2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.780974:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495781248:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35701us (36111us total) trans 4294972799 rc 0/0 00000100:00100000:1.0:1789760729.780988:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5511 00000100:00100000:1.0:1789760729.781002:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714022489406 time 714065963278 state 0x2 ref 0 00000100:00100000:1.0:1789760729.784746:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760729.785171:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495781376 00000100:00100000:3.0:1789760729.785186:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495781376 00000100:00100000:3.0:1789760729.785268:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714022489406 time 714065963278 state 0x0 ref 1 00000100:00100000:3.0:1789760729.785338:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714066223694 time 714065963278 state 0x1 ref 1 00000100:00100000:3.0:1789760729.785371:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714066223694 time 714106087820 state 0x0 ref 1 00000100:00100000:3.0:1789760729.785387:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370bd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495781376:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760729.785468:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22458368, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:3.0:1789760729.785478:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545db9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.785498:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760729.785503:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.785511:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545db9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.785531:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545db9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760729.819305:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972800 is committed 00002000:00010000:1.0:1789760729.819424:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22458368 -> 22462464 00010000:00010000:1.0:1789760729.819440:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545db9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.819452:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545db9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.819461:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545db9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.819468:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760729.819473:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545db9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.819481:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545db9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.819506:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545db9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.819513:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545db9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.819606:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370bd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495781376:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34217us (34871us total) trans 4294972800 rc 0/0 00000100:00100000:1.0:1789760729.819616:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5512 00000100:00100000:1.0:1789760729.819626:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714066223694 time 714106087820 state 0x2 ref 0 00000100:00100000:3.0:1789760729.824425:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760729.825318:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495781504 00000100:00100000:0.0:1789760729.825330:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495781504 00000100:00100000:0.0:1789760729.825389:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714066223694 time 714106087820 state 0x0 ref 1 00000100:00100000:0.0:1789760729.825407:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714106348236 time 714106087820 state 0x1 ref 1 00000100:00100000:0.0:1789760729.825437:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714106348236 time 714146153957 state 0x0 ref 1 00000100:00100000:0.0:1789760729.825452:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495781504:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760729.825558:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22462464, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:0.0:1789760729.825571:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545dc0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.825589:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760729.825594:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.825601:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545dc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.825614:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545dc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.856832:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972801 is committed 00002000:00010000:0.0:1789760729.856953:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22462464 -> 22466560 00010000:00010000:0.0:1789760729.856969:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545dc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.856981:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545dc0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.856990:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545dc0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.856997:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760729.857001:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545dc0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.857009:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545dc0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.857028:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545dc0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.857036:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545dc0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.857157:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495781504:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31681us (32712us total) trans 4294972801 rc 0/0 00000100:00100000:0.0:1789760729.857169:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5513 00000100:00100000:0.0:1789760729.857179:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714106348236 time 714146153957 state 0x2 ref 0 00000100:00100000:2.0:1789760729.862348:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760729.862460:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495781632 00000100:00100000:1.0:1789760729.862469:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495781632 00000100:00100000:1.0:1789760729.862526:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714106348236 time 714146153957 state 0x0 ref 1 00000100:00100000:1.0:1789760729.862543:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714146414373 time 714146153957 state 0x1 ref 1 00000100:00100000:1.0:1789760729.862568:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714146414373 time 714183284751 state 0x0 ref 1 00000100:00100000:1.0:1789760729.862580:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495781632:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760729.862629:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22466560, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:1.0:1789760729.862638:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545dc7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.862659:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760729.862663:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.862670:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545dc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.862684:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545dc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760729.895053:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22466560 -> 22470656 00010000:00010000:1.0:1789760729.895075:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545dc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.895089:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545dc7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.895098:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545dc7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.895105:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760729.895110:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545dc7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.895118:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545dc7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.895151:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545dc7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.895160:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545dc7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.895257:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495781632:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32674us (32914us total) trans 4294972802 rc 0/0 00000100:00100000:1.0:1789760729.895268:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5514 00000100:00100000:1.0:1789760729.895318:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714146414373 time 714183284751 state 0x2 ref 0 00000001:00080000:1.0:1789760729.895361:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972802 is committed 00000100:00100000:1.0:1789760729.899262:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760729.899378:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495781760 00000100:00100000:3.0:1789760729.899389:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495781760 00000100:00100000:3.0:1789760729.899461:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714146414373 time 714183284751 state 0x0 ref 1 00000100:00100000:3.0:1789760729.899477:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714183545167 time 714183284751 state 0x1 ref 1 00000100:00100000:3.0:1789760729.899512:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714183545167 time 714220222875 state 0x0 ref 1 00000100:00100000:2.0:1789760729.899617:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495781760:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760729.899681:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22470656, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:2.0:1789760729.899690:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545dce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.899705:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760729.899710:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.899716:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545dce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.899731:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545dce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760729.945402:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972803 is committed 00002000:00010000:1.0:1789760729.945520:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22470656 -> 22474752 00010000:00010000:1.0:1789760729.945535:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545dce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.945547:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545dce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.945558:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545dce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.945565:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760729.945570:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545dce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.945579:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545dce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.945611:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545dce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.945620:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545dce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.945713:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495781760:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46185us (46454us total) trans 4294972803 rc 0/0 00000100:00100000:1.0:1789760729.945723:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5515 00000100:00100000:1.0:1789760729.945733:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714183545167 time 714220222875 state 0x2 ref 0 00000100:00100000:3.0:1789760729.949971:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760729.950263:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495781888 00000100:00100000:0.0:1789760729.950273:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495781888 00000100:00100000:0.0:1789760729.950362:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714183545167 time 714220222875 state 0x0 ref 1 00000100:00100000:0.0:1789760729.950377:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714220483291 time 714220222875 state 0x1 ref 1 00000100:00100000:0.0:1789760729.950398:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714220483291 time 714271115317 state 0x0 ref 1 00000100:00100000:0.0:1789760729.950410:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495781888:12345-192.168.203.9@tcp:4:dd.0 00010000:00010000:1.0:1789760729.950779:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760729.950806:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22474752, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:0.0:1789760729.950816:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545dd5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.950834:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760729.950838:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.950844:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545dd5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.950865:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545dd5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760729.986353:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22474752 -> 22478848 00010000:00010000:3.0:1789760729.986376:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545dd5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.986390:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545dd5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.986399:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545dd5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.986406:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760729.986411:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545dd5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.986424:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545dd5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.986445:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545dd5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.986453:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545dd5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.986680:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495781888:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36266us (36715us total) trans 4294972804 rc 0/0 00000100:00100000:3.0:1789760729.986692:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5516 00000100:00100000:3.0:1789760729.986702:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714220483291 time 714271115317 state 0x2 ref 0 00000001:00080000:1.0:1789760729.987308:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972804 is committed 00000100:00100000:2.0:1789760730.000665:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760730.000794:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495782016 00000100:00100000:1.0:1789760730.000804:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495782016 00000100:00100000:1.0:1789760730.000861:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714220483291 time 714271115317 state 0x0 ref 1 00000100:00100000:1.0:1789760730.000876:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714271375733 time 714271115317 state 0x1 ref 1 00000100:00100000:1.0:1789760730.000912:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714271375733 time 714321628563 state 0x0 ref 1 00000100:00100000:1.0:1789760730.000925:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495782016:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760730.000975:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22478848, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:1.0:1789760730.000985:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545ddc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.001006:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760730.001011:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.001017:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545ddc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.001032:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545ddc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760730.038199:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972805 is committed 00002000:00010000:1.0:1789760730.038708:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22478848 -> 22482944 00010000:00010000:1.0:1789760730.038725:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545ddc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.038737:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545ddc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.038745:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545ddc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.038753:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760730.038757:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545ddc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.038766:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545ddc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.038786:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545ddc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.038795:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545ddc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.038933:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495782016:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38006us (38273us total) trans 4294972805 rc 0/0 00000100:00100000:1.0:1789760730.038944:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5517 00000100:00100000:1.0:1789760730.038954:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714271375733 time 714321628563 state 0x2 ref 0 00000100:00100000:2.0:1789760730.041703:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760730.041803:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495782144 00000100:00100000:1.0:1789760730.041813:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495782144 00000100:00100000:1.0:1789760730.041873:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714271375733 time 714321628563 state 0x0 ref 1 00000100:00100000:1.0:1789760730.041888:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714321888979 time 714321628563 state 0x1 ref 1 00000100:00100000:1.0:1789760730.042016:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714321888979 time 714362732377 state 0x0 ref 1 00000100:00100000:1.0:1789760730.042030:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495782144:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760730.042077:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22482944, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:1.0:1789760730.042086:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354545de3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.042101:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760730.042106:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.042201:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354545de3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.042223:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354545de3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760730.103445:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22482944 -> 22487040 00010000:00010000:0.0:1789760730.103465:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354545de3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.103478:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354545de3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.103487:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354545de3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.103495:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760730.103500:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354545de3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.103508:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354545de3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.103810:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354545de3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.103822:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354545de3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.103943:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495782144:12345-192.168.203.9@tcp:4:dd.0 Request processed in 61910us (62242us total) trans 4294972806 rc 0/0 00000100:00100000:0.0:1789760730.103954:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5518 00000100:00100000:0.0:1789760730.103964:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714321888979 time 714362732377 state 0x2 ref 0 00000001:00080000:2.0:1789760730.104461:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972806 is committed 00000100:00100000:3.0:1789760730.108882:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760730.108943:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495782272 00000100:00100000:0.0:1789760730.108953:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495782272 00000100:00100000:0.0:1789760730.109008:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714321888979 time 714362732377 state 0x0 ref 1 00000100:00100000:0.0:1789760730.109035:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714362992793 time 714362732377 state 0x1 ref 1 00000100:00100000:0.0:1789760730.109079:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714362992793 time 714429796216 state 0x0 ref 1 00000100:00100000:0.0:1789760730.109093:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495782272:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760730.109150:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22487040, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:0.0:1789760730.109158:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545dea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.109175:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760730.109180:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760730.109188:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545dea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.109203:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545dea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760730.155737:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972807 is committed 00002000:00010000:2.0:1789760730.156158:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22487040 -> 22491136 00010000:00010000:2.0:1789760730.156173:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545dea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.156191:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545dea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.156200:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545dea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.156207:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760730.156212:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545dea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.156221:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545dea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.156242:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545dea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.156250:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354545dea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.156380:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495782272:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47285us (47505us total) trans 4294972807 rc 0/0 00000100:00100000:2.0:1789760730.156390:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5519 00000100:00100000:2.0:1789760730.156399:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714362992793 time 714429796216 state 0x2 ref 0 00000100:00100000:3.0:1789760730.164020:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760730.164093:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495782400 00000100:00100000:1.0:1789760730.164104:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495782400 00000100:00100000:1.0:1789760730.164168:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714362992793 time 714429796216 state 0x0 ref 1 00000100:00100000:1.0:1789760730.164183:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714430056632 time 714429796216 state 0x1 ref 1 00000100:00100000:1.0:1789760730.164216:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714430056632 time 714484932131 state 0x0 ref 1 00000100:00100000:1.0:1789760730.164229:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495782400:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760730.164276:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22491136, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:1.0:1789760730.164314:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545df1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.164338:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760730.164342:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.164348:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545df1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.164360:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545df1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760730.197036:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972808 is committed 00002000:00010000:2.0:1789760730.197216:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22491136 -> 22495232 00010000:00010000:2.0:1789760730.197232:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545df1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.197245:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545df1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.197255:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545df1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.197262:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760730.197268:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545df1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.197306:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545df1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.197329:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545df1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.197338:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354545df1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.197434:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495782400:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33202us (33420us total) trans 4294972808 rc 0/0 00000100:00100000:2.0:1789760730.197444:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5520 00000100:00100000:2.0:1789760730.197455:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714430056632 time 714484932131 state 0x2 ref 0 00000100:00100000:1.0:1789760730.202941:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760730.203023:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495782528 00000100:00100000:0.0:1789760730.203032:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495782528 00000100:00100000:0.0:1789760730.203089:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714430056632 time 714484932131 state 0x0 ref 1 00000100:00100000:0.0:1789760730.203105:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714485192547 time 714484932131 state 0x1 ref 1 00000100:00100000:0.0:1789760730.203180:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714485192547 time 714523895997 state 0x0 ref 1 00000100:00100000:0.0:1789760730.203193:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495782528:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760730.203243:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22495232, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:0.0:1789760730.203252:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545df8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.203269:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760730.203273:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760730.203314:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545df8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.203330:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545df8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760730.233962:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22495232 -> 22499328 00010000:00010000:0.0:1789760730.233983:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545df8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.233996:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545df8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.234006:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545df8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.234014:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760730.234019:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545df8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.234029:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545df8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.234048:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545df8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.234057:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545df8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.234172:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495782528:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30976us (31235us total) trans 4294972809 rc 0/0 00000100:00100000:0.0:1789760730.234184:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5521 00000100:00100000:0.0:1789760730.234194:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714485192547 time 714523895997 state 0x2 ref 0 00000001:00080000:1.0:1789760730.234732:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972809 is committed 00000100:00100000:0.0:1789760730.237922:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760730.238007:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495782656 00000100:00100000:1.0:1789760730.238016:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495782656 00000100:00100000:1.0:1789760730.238070:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714485192547 time 714523895997 state 0x0 ref 1 00000100:00100000:1.0:1789760730.238085:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714524156413 time 714523895997 state 0x1 ref 1 00000100:00100000:1.0:1789760730.238122:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714524156413 time 714558838034 state 0x0 ref 1 00000100:00100000:1.0:1789760730.238144:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495782656:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760730.238189:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22499328, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:1.0:1789760730.238198:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545dff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.238213:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760730.238218:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.238224:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545dff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.238238:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545dff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760730.266301:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972810 is committed 00002000:00010000:1.0:1789760730.266441:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22499328 -> 22503424 00010000:00010000:1.0:1789760730.266528:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545dff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.266542:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545dff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.266551:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545dff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.266559:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760730.266564:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545dff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.266572:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545dff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.266592:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545dff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.266600:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354545dff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.266731:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495782656:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28585us (28815us total) trans 4294972810 rc 0/0 00000100:00100000:1.0:1789760730.266742:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5522 00000100:00100000:1.0:1789760730.266753:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714524156413 time 714558838034 state 0x2 ref 0 00000100:00100000:2.0:1789760730.271175:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760730.271371:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495782784 00000100:00100000:1.0:1789760730.271381:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495782784 00000100:00100000:1.0:1789760730.271434:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714524156413 time 714558838034 state 0x0 ref 1 00000100:00100000:1.0:1789760730.271512:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714559098450 time 714558838034 state 0x1 ref 1 00000100:00100000:1.0:1789760730.271585:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714559098450 time 714592300580 state 0x0 ref 1 00000100:00100000:1.0:1789760730.271598:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495782784:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760730.271649:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22503424, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:1.0:1789760730.271659:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545e06 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.271676:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760730.271681:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.271687:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545e06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.271701:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545e06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760730.329961:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972811 is committed 00002000:00010000:1.0:1789760730.330074:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22503424 -> 22507520 00010000:00010000:1.0:1789760730.330089:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545e06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.330116:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545e06 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.330138:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545e06 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.330145:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760730.330151:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545e06 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.330160:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545e06 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.330179:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545e06 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.330186:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545e06 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.330277:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495782784:12345-192.168.203.9@tcp:4:dd.0 Request processed in 58677us (59174us total) trans 4294972811 rc 0/0 00000100:00100000:1.0:1789760730.330317:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5523 00000100:00100000:1.0:1789760730.330327:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714559098450 time 714592300580 state 0x2 ref 0 00000100:00100000:2.0:1789760730.335699:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760730.335811:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495782912 00000100:00100000:0.0:1789760730.335821:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495782912 00000100:00100000:0.0:1789760730.335878:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714559098450 time 714592300580 state 0x0 ref 1 00000100:00100000:0.0:1789760730.335894:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714592560996 time 714592300580 state 0x1 ref 1 00000100:00100000:0.0:1789760730.335977:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714592560996 time 714656692530 state 0x0 ref 1 00000100:00100000:0.0:1789760730.335990:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495782912:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760730.336038:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22507520, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:0.0:1789760730.336047:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545e0d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.336063:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760730.336067:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760730.336073:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545e0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.336086:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545e0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760730.373669:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972812 is committed 00002000:00010000:1.0:1789760730.373777:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22507520 -> 22511616 00010000:00010000:1.0:1789760730.373791:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545e0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.373803:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545e0d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.373812:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545e0d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.373824:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760730.373829:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545e0d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.373837:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545e0d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.373856:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545e0d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.373864:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354545e0d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.373992:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495782912:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37999us (38297us total) trans 4294972812 rc 0/0 00000100:00100000:1.0:1789760730.374002:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5524 00000100:00100000:1.0:1789760730.374012:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714592560996 time 714656692530 state 0x2 ref 0 00000100:00100000:2.0:1789760730.377987:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760730.378081:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495783040 00000100:00100000:1.0:1789760730.378091:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495783040 00000100:00100000:1.0:1789760730.378214:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714592560996 time 714656692530 state 0x0 ref 1 00000100:00100000:1.0:1789760730.378228:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714656952946 time 714656692530 state 0x1 ref 1 00000100:00100000:1.0:1789760730.382514:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714656952946 time 714703217826 state 0x0 ref 1 00000100:00100000:1.0:1789760730.382551:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495783040:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760730.382610:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22511616, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:1.0:1789760730.382620:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545e14 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.382637:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760730.382642:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.382649:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545e14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.382663:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545e14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.399901:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff92900c6a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520304896:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760730.399973:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760730.400041:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520304896 00000100:00100000:1.0:1789760730.400048:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520304896 00000100:00100000:1.0:1789760730.400092:0:6711:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 195 00000100:00100000:1.0:1789760730.400100:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520304896:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760730.400140:0:6711:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528154624 pending 1703936 free 3835691008 avail 3796893696 00000100:00100000:1.0:1789760730.400177:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520304896:12345-0@lo:13:osp-pre-0-0.0 Request processed in 78us (203us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760730.400184:0:6711:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 195 00000100:00080000:1.0:1789760730.400201:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760730.400234:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff92900c6a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520304896:0@lo:13:osp-pre-0-0.0 00000001:00080000:2.0:1789760730.412987:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972813 is committed 00002000:00010000:0.0:1789760730.414044:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22511616 -> 22515712 00010000:00010000:0.0:1789760730.414066:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545e14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.414082:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545e14 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.414096:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545e14 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.414104:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760730.414109:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545e14 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.414118:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545e14 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.414151:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545e14 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.414160:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545e14 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.414352:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495783040:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31812us (36368us total) trans 4294972813 rc 0/0 00000100:00100000:0.0:1789760730.414362:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5525 00000100:00100000:0.0:1789760730.414371:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714656952946 time 714703217826 state 0x2 ref 0 00000100:00100000:2.0:1789760730.418422:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760730.418953:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495783168 00000100:00100000:3.0:1789760730.418964:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495783168 00000100:00100000:3.0:1789760730.419015:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714656952946 time 714703217826 state 0x0 ref 1 00000100:00100000:3.0:1789760730.419029:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714703478242 time 714703217826 state 0x1 ref 1 00000100:00100000:3.0:1789760730.419055:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714703478242 time 714739772512 state 0x0 ref 1 00000100:00100000:3.0:1789760730.419065:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495783168:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760730.419111:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22515712, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:3.0:1789760730.419119:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e1b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.419146:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760730.419151:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760730.419157:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.419170:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760730.448966:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972814 is committed 00002000:00010000:3.0:1789760730.449027:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22515712 -> 22519808 00010000:00010000:3.0:1789760730.449042:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.449054:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e1b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.449063:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e1b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.449069:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760730.449074:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e1b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.449084:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e1b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.449113:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e1b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.449121:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e1b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.449219:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495783168:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30150us (30801us total) trans 4294972814 rc 0/0 00000100:00100000:3.0:1789760730.449230:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5526 00000100:00100000:3.0:1789760730.449240:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714703478242 time 714739772512 state 0x2 ref 0 00000100:00100000:1.0:1789760730.453139:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760730.453234:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495783296 00000100:00100000:0.0:1789760730.453246:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495783296 00000100:00100000:0.0:1789760730.453350:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714703478242 time 714739772512 state 0x0 ref 1 00000100:00100000:0.0:1789760730.453367:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714740032928 time 714739772512 state 0x1 ref 1 00000100:00100000:0.0:1789760730.453393:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714740032928 time 714774109793 state 0x0 ref 1 00000100:00100000:0.0:1789760730.453405:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495783296:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760730.453465:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22519808, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:0.0:1789760730.453474:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545e22 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.453490:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760730.453495:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760730.453502:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545e22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.453516:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545e22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760730.484133:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22519808 -> 22523904 00010000:00010000:1.0:1789760730.484154:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545e22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.484167:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545e22 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.484176:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545e22 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.484183:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760730.484188:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545e22 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.484198:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545e22 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.484217:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545e22 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.484225:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354545e22 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.484357:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495783296:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30948us (31229us total) trans 4294972815 rc 0/0 00000100:00100000:1.0:1789760730.484368:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5527 00000100:00100000:1.0:1789760730.484377:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714740032928 time 714774109793 state 0x2 ref 0 00000001:00080000:2.0:1789760730.484844:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972815 is committed 00000100:00100000:3.0:1789760730.488524:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760730.488750:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495783424 00000100:00100000:0.0:1789760730.488759:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495783424 00000100:00100000:0.0:1789760730.488828:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714740032928 time 714774109793 state 0x0 ref 1 00000100:00100000:0.0:1789760730.488846:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714774370209 time 714774109793 state 0x1 ref 1 00000100:00100000:0.0:1789760730.488878:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714774370209 time 714809594633 state 0x0 ref 1 00000100:00100000:0.0:1789760730.488892:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495783424:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760730.488938:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22523904, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:0.0:1789760730.488947:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545e29 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.488962:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760730.488967:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760730.488974:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545e29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.488987:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545e29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760730.523506:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22523904 -> 22528000 00010000:00010000:1.0:1789760730.523525:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545e29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.523537:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545e29 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.523545:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545e29 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.523552:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760730.523557:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545e29 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.523565:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545e29 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.523583:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545e29 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.523591:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545e29 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.523687:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495783424:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34793us (35166us total) trans 4294972816 rc 0/0 00000100:00100000:1.0:1789760730.523701:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5528 00000100:00100000:1.0:1789760730.523712:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714774370209 time 714809594633 state 0x2 ref 0 00000001:00080000:3.0:1789760730.525608:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972816 is committed 00000100:00100000:0.0:1789760730.527544:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760730.527772:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495783552 00000100:00100000:3.0:1789760730.527781:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495783552 00000100:00100000:3.0:1789760730.527834:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714774370209 time 714809594633 state 0x0 ref 1 00000100:00100000:3.0:1789760730.527849:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714809855049 time 714809594633 state 0x1 ref 1 00000100:00100000:3.0:1789760730.527954:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714809855049 time 714848668927 state 0x0 ref 1 00000100:00100000:3.0:1789760730.527968:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495783552:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760730.528020:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22528000, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:3.0:1789760730.528029:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e30 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.528046:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760730.528051:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760730.528057:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.528071:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760730.561266:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22528000 -> 22532096 00010000:00010000:3.0:1789760730.561304:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.561318:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e30 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.561328:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e30 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.561335:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760730.561341:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e30 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.561350:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e30 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.561371:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e30 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.561379:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e30 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.561478:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495783552:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33509us (33938us total) trans 4294972817 rc 0/0 00000100:00100000:3.0:1789760730.561489:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5529 00000100:00100000:3.0:1789760730.561499:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714809855049 time 714848668927 state 0x2 ref 0 00000001:00080000:1.0:1789760730.561959:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972817 is committed 00000100:00100000:0.0:1789760730.565531:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760730.565718:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495783680 00000100:00100000:1.0:1789760730.565728:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495783680 00000100:00100000:1.0:1789760730.565795:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714809855049 time 714848668927 state 0x0 ref 1 00000100:00100000:1.0:1789760730.565812:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714848929343 time 714848668927 state 0x1 ref 1 00000100:00100000:1.0:1789760730.565840:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714848929343 time 714886557251 state 0x0 ref 1 00000100:00100000:1.0:1789760730.565853:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495783680:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760730.565904:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22532096, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:1.0:1789760730.565913:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e37 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.565928:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760730.565932:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.565938:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.565952:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760730.601041:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972818 is committed 00002000:00010000:1.0:1789760730.601162:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22532096 -> 22536192 00010000:00010000:1.0:1789760730.601178:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.601190:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e37 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.601198:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e37 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.601205:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760730.601211:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e37 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.601220:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e37 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.601240:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e37 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.601248:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e37 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.601597:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495783680:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35737us (36065us total) trans 4294972818 rc 0/0 00000100:00100000:2.0:1789760730.601608:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5530 00000100:00100000:2.0:1789760730.601617:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714848929343 time 714886557251 state 0x2 ref 0 00000100:00100000:2.0:1789760730.607053:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760730.607338:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495783808 00000100:00100000:1.0:1789760730.607348:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495783808 00000100:00100000:1.0:1789760730.607405:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714848929343 time 714886557251 state 0x0 ref 1 00000100:00100000:1.0:1789760730.607421:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714886817667 time 714886557251 state 0x1 ref 1 00000100:00100000:2.0:1789760730.607735:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714886817667 time 714928449591 state 0x0 ref 1 00000100:00100000:2.0:1789760730.607751:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495783808:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760730.607802:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22536192, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:2.0:1789760730.607812:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545e3e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.607827:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760730.607832:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.607837:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545e3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.607851:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545e3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760730.645876:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972819 is committed 00002000:00010000:3.0:1789760730.646079:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22536192 -> 22540288 00010000:00010000:3.0:1789760730.646094:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545e3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.646179:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545e3e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.646189:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545e3e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.646197:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760730.646202:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545e3e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.646217:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545e3e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.646237:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545e3e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.646244:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354545e3e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.646372:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495783808:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38621us (39324us total) trans 4294972819 rc 0/0 00000100:00100000:3.0:1789760730.646383:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5531 00000100:00100000:3.0:1789760730.646392:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714886817667 time 714928449591 state 0x2 ref 0 00000100:00100000:0.0:1789760730.650660:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760730.650920:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495783936 00000100:00100000:1.0:1789760730.650930:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495783936 00000100:00100000:1.0:1789760730.650983:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714886817667 time 714928449591 state 0x0 ref 1 00000100:00100000:1.0:1789760730.650998:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714928710007 time 714928449591 state 0x1 ref 1 00000100:00100000:1.0:1789760730.651028:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714928710007 time 714971745518 state 0x0 ref 1 00000100:00100000:1.0:1789760730.651041:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495783936:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760730.651092:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22540288, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:1.0:1789760730.651271:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e45 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.651314:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760730.651319:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.651325:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.651338:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760730.693688:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22540288 -> 22544384 00002000:00010000:1.0:1789760730.693698:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760729 -> 1789760730 00002000:00010000:1.0:1789760730.693701:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760729 -> 1789760730 00000001:00080000:2.0:1789760730.693701:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972820 is committed 00010000:00010000:1.0:1789760730.693719:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.693740:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e45 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.693749:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e45 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.693756:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760730.693761:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e45 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.693769:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e45 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.693792:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e45 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.693799:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e45 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.693904:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495783936:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42860us (43248us total) trans 4294972820 rc 0/0 00000100:00100000:1.0:1789760730.693925:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5532 00000100:00100000:1.0:1789760730.693935:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714928710007 time 714971745518 state 0x2 ref 0 00000100:00100000:2.0:1789760730.703973:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760730.704151:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495784064 00000100:00100000:0.0:1789760730.704160:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495784064 00000100:00100000:0.0:1789760730.704213:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714928710007 time 714971745518 state 0x0 ref 1 00000100:00100000:0.0:1789760730.704228:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714972005934 time 714971745518 state 0x1 ref 1 00000100:00100000:0.0:1789760730.704259:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714972005934 time 715024975683 state 0x0 ref 1 00000100:00100000:0.0:1789760730.704314:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495784064:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760730.704364:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22544384, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb007 00010000:00010000:0.0:1789760730.704372:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545e4c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.704392:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760730.704397:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760730.704404:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545e4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.704419:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545e4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760730.763235:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972821 is committed 00002000:00010000:0.0:1789760730.763411:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22544384 -> 22548480 00010000:00010000:0.0:1789760730.763426:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545e4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.763439:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545e4c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.763447:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545e4c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.763454:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760730.763512:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545e4c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.763522:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545e4c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.763541:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545e4c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.763549:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354545e4c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.763672:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495784064:12345-192.168.203.9@tcp:4:dd.0 Request processed in 59388us (59702us total) trans 4294972821 rc 0/0 00000100:00100000:0.0:1789760730.763683:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5533 00000100:00100000:0.0:1789760730.763693:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714972005934 time 715024975683 state 0x2 ref 0 00000100:00100000:2.0:1789760730.775630:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760730.775872:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495784192 00000100:00100000:3.0:1789760730.775881:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495784192 00000100:00100000:3.0:1789760730.775936:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 714972005934 time 715024975683 state 0x0 ref 1 00000100:00100000:3.0:1789760730.775951:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715025236099 time 715024975683 state 0x1 ref 1 00000100:00100000:3.0:1789760730.775983:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715025236099 time 715096699660 state 0x0 ref 1 00000100:00100000:3.0:1789760730.775995:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495784192:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760730.776041:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22548480, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb007 00010000:00010000:3.0:1789760730.776050:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e53 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.776067:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760730.776072:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760730.776079:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.776093:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760730.846609:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972822 is committed 00002000:00010000:3.0:1789760730.846673:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22548480 -> 22552576 00010000:00010000:3.0:1789760730.846687:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.846701:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e53 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.846710:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e53 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.846718:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760730.846723:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e53 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.846731:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e53 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.846752:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e53 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.846761:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e53 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.846868:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495784192:12345-192.168.203.9@tcp:4:dd.0 Request processed in 70870us (71243us total) trans 4294972822 rc 0/0 00000100:00100000:3.0:1789760730.846878:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5534 00000100:00100000:3.0:1789760730.846887:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715025236099 time 715096699660 state 0x2 ref 0 00000100:00100000:2.0:1789760730.855587:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760730.855854:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495784320 00000100:00100000:1.0:1789760730.855864:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495784320 00000100:00100000:1.0:1789760730.855923:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715025236099 time 715096699660 state 0x0 ref 1 00000100:00100000:1.0:1789760730.855938:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715096960076 time 715096699660 state 0x1 ref 1 00000100:00100000:1.0:1789760730.857005:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715096960076 time 715177712060 state 0x0 ref 1 00000100:00100000:1.0:1789760730.857034:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495784320:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760730.857111:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22552576, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb007 00010000:00010000:1.0:1789760730.857128:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e5a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.857145:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760730.857149:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.857154:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.857168:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760730.901403:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972823 is committed 00002000:00010000:3.0:1789760730.901623:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22552576 -> 22556672 00010000:00010000:3.0:1789760730.901641:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.901654:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e5a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.901664:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e5a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.901672:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760730.901676:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e5a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.901691:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e5a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.901712:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e5a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.901721:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545e5a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.901867:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495784320:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44840us (46284us total) trans 4294972823 rc 0/0 00000100:00100000:3.0:1789760730.901880:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5535 00000100:00100000:3.0:1789760730.901891:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715096960076 time 715177712060 state 0x2 ref 0 00000100:00100000:0.0:1789760730.912175:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760730.912580:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495784448 00000100:00100000:2.0:1789760730.912590:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495784448 00000100:00100000:2.0:1789760730.912643:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715096960076 time 715177712060 state 0x0 ref 1 00000100:00100000:2.0:1789760730.912659:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715177972476 time 715177712060 state 0x1 ref 1 00000100:00100000:2.0:1789760730.913099:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715177972476 time 715233813497 state 0x0 ref 1 00000100:00100000:2.0:1789760730.913113:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495784448:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760730.913180:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22556672, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb007 00010000:00010000:2.0:1789760730.913189:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545e61 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.913203:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760730.913206:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.913212:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545e61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.913226:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545e61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760730.948096:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972824 is committed 00002000:00010000:0.0:1789760730.948390:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22556672 -> 22560768 00010000:00010000:0.0:1789760730.948406:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545e61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.948421:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545e61 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.948429:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545e61 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.948436:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760730.948441:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545e61 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.948450:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545e61 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.948470:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545e61 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.948478:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545e61 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.948576:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495784448:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35460us (36413us total) trans 4294972824 rc 0/0 00000100:00100000:0.0:1789760730.948587:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5536 00000100:00100000:0.0:1789760730.948597:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715177972476 time 715233813497 state 0x2 ref 0 00000100:00100000:2.0:1789760730.954116:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760730.954254:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495784576 00000100:00100000:3.0:1789760730.954264:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495784576 00000100:00100000:3.0:1789760730.954360:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715177972476 time 715233813497 state 0x0 ref 1 00000100:00100000:3.0:1789760730.954374:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715234073913 time 715233813497 state 0x1 ref 1 00000100:00100000:3.0:1789760730.954396:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715234073913 time 715275113708 state 0x0 ref 1 00000100:00100000:3.0:1789760730.954408:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495784576:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760730.954453:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22560768, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb007 00010000:00010000:3.0:1789760730.954462:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e68 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.954478:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760730.954483:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760730.954489:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.954502:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.974452:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760730.997629:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22560768 -> 22564864 00010000:00010000:3.0:1789760730.997649:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.997664:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e68 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.997672:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e68 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.997680:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760730.997684:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e68 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.997693:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e68 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.997718:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e68 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.997725:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e68 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.997817:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495784576:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43407us (43715us total) trans 4294972825 rc 0/0 00000100:00100000:3.0:1789760730.997828:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5537 00000100:00100000:3.0:1789760730.997837:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715234073913 time 715275113708 state 0x2 ref 0 00000001:00080000:0.0:1789760730.999367:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972825 is committed 00000100:00100000:2.0:1789760731.001145:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760731.001265:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495784704 00000100:00100000:3.0:1789760731.001275:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495784704 00000100:00100000:3.0:1789760731.001363:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715234073913 time 715275113708 state 0x0 ref 1 00000100:00100000:3.0:1789760731.001378:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715275374124 time 715275113708 state 0x1 ref 1 00000100:00100000:3.0:1789760731.001396:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715275374124 time 715322114402 state 0x0 ref 1 00000100:00100000:3.0:1789760731.001407:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495784704:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760731.001520:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22564864, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb007 00010000:00010000:3.0:1789760731.001529:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545e6f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.001543:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760731.001548:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.001554:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545e6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.001569:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545e6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760731.001996:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760731.028049:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972826 is committed 00002000:00010000:0.0:1789760731.028217:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22564864 -> 22568960 00010000:00010000:0.0:1789760731.028233:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545e6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.028246:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545e6f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.028256:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545e6f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.028263:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760731.028268:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545e6f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.028301:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545e6f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.028322:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545e6f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.028330:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354545e6f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760731.028463:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495784704:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27051us (27319us total) trans 4294972826 rc 0/0 00000100:00100000:0.0:1789760731.028472:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5538 00000100:00100000:0.0:1789760731.028481:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715275374124 time 715322114402 state 0x2 ref 0 00000100:00100000:1.0:1789760731.035856:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760731.036171:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495784832 00000100:00100000:0.0:1789760731.036181:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495784832 00000100:00100000:0.0:1789760731.036238:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715275374124 time 715322114402 state 0x0 ref 1 00000100:00100000:0.0:1789760731.036256:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715322374818 time 715322114402 state 0x1 ref 1 00000100:00100000:0.0:1789760731.036350:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715322374818 time 715357063621 state 0x0 ref 1 00000100:00100000:0.0:1789760731.036364:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495784832:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760731.036413:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22568960, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb007 00010000:00010000:0.0:1789760731.036423:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545e76 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.036439:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760731.036443:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760731.036450:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545e76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.036464:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545e76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760731.078690:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22568960 -> 22573056 00010000:00010000:0.0:1789760731.078718:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545e76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.078731:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545e76 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.078741:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545e76 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.078748:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760731.078753:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545e76 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.078762:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545e76 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.078781:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545e76 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.078790:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354545e76 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760731.078882:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495784832:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42516us (43031us total) trans 4294972827 rc 0/0 00000100:00100000:0.0:1789760731.078893:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5539 00000100:00100000:0.0:1789760731.078901:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715322374818 time 715357063621 state 0x2 ref 0 00000001:00080000:2.0:1789760731.079044:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972827 is committed 00000100:00100000:2.0:1789760731.082916:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760731.083058:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495784960 00000100:00100000:3.0:1789760731.083067:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495784960 00000100:00100000:3.0:1789760731.083140:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715322374818 time 715357063621 state 0x0 ref 1 00000100:00100000:3.0:1789760731.083155:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715357324037 time 715357063621 state 0x1 ref 1 00000100:00100000:3.0:1789760731.083185:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715357324037 time 715403901661 state 0x0 ref 1 00000100:00100000:3.0:1789760731.083197:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495784960:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760731.083242:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22573056, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb007 00010000:00010000:3.0:1789760731.083251:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e7d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.083266:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760731.083270:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.083276:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.083323:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760731.117701:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22573056 -> 22577152 00010000:00010000:3.0:1789760731.117723:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.117737:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e7d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.117747:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e7d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.117755:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760731.117760:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e7d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.117769:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e7d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.117788:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e7d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.117795:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e7d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.117883:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495784960:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34684us (34971us total) trans 4294972828 rc 0/0 00000100:00100000:3.0:1789760731.117898:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5540 00000100:00100000:3.0:1789760731.117907:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715357324037 time 715403901661 state 0x2 ref 0 00000001:00080000:1.0:1789760731.118506:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972828 is committed 00000100:00100000:1.0:1789760731.123830:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760731.124179:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495785088 00000100:00100000:0.0:1789760731.124189:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495785088 00000100:00100000:0.0:1789760731.124248:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715357324037 time 715403901661 state 0x0 ref 1 00000100:00100000:0.0:1789760731.124264:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715404162077 time 715403901661 state 0x1 ref 1 00000100:00100000:0.0:1789760731.124361:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715404162077 time 715445077401 state 0x0 ref 1 00000100:00100000:0.0:1789760731.124374:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495785088:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760731.124423:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22577152, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb007 00010000:00010000:0.0:1789760731.124432:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545e84 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.124448:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760731.124454:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760731.124461:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545e84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.124479:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545e84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760731.163080:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22577152 -> 22581248 00010000:00010000:1.0:1789760731.163106:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545e84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.163128:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545e84 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.163138:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545e84 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.163145:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760731.163151:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545e84 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.163161:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545e84 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.163182:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545e84 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.163190:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354545e84 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.163307:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495785088:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38903us (39455us total) trans 4294972829 rc 0/0 00000100:00100000:1.0:1789760731.163319:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5541 00000100:00100000:1.0:1789760731.163329:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715404162077 time 715445077401 state 0x2 ref 0 00000001:00080000:2.0:1789760731.163799:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972829 is committed 00000100:00100000:0.0:1789760731.167917:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760731.168068:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495785216 00000100:00100000:3.0:1789760731.168077:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495785216 00000100:00100000:3.0:1789760731.168387:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715404162077 time 715445077401 state 0x0 ref 1 00000100:00100000:3.0:1789760731.168403:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715445337817 time 715445077401 state 0x1 ref 1 00000100:00100000:3.0:1789760731.168424:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715445337817 time 715489141356 state 0x0 ref 1 00000100:00100000:3.0:1789760731.168436:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495785216:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760731.168601:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22581248, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb007 00010000:00010000:3.0:1789760731.168611:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e8b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.168627:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760731.168632:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.168639:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.168654:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760731.193413:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972830 is committed 00002000:00010000:2.0:1789760731.193620:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22581248 -> 22585344 00010000:00010000:2.0:1789760731.193635:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.193648:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e8b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.193657:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.193664:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760731.193668:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.193678:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e8b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.193698:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e8b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.193706:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545e8b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.193798:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495785216:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25359us (25885us total) trans 4294972830 rc 0/0 00000100:00100000:2.0:1789760731.193808:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5542 00000100:00100000:2.0:1789760731.193818:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715445337817 time 715489141356 state 0x2 ref 0 00000100:00100000:0.0:1789760731.196892:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760731.197079:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495785344 00000100:00100000:3.0:1789760731.197090:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495785344 00000100:00100000:3.0:1789760731.197395:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715445337817 time 715489141356 state 0x0 ref 1 00000100:00100000:3.0:1789760731.197410:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715489401772 time 715489141356 state 0x1 ref 1 00000100:00100000:3.0:1789760731.197451:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715489401772 time 715518159702 state 0x0 ref 1 00000100:00100000:3.0:1789760731.197465:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495785344:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760731.197784:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22585344, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb007 00010000:00010000:3.0:1789760731.197793:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545e92 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.197809:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760731.197814:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.197821:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545e92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.197835:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545e92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760731.232441:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972831 is committed 00002000:00010000:3.0:1789760731.232571:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22585344 -> 22589440 00010000:00010000:3.0:1789760731.232589:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545e92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.232602:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545e92 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.232610:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545e92 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.232617:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760731.232622:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545e92 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.232631:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545e92 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.232649:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545e92 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.232658:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545e92 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.232746:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495785344:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35278us (35859us total) trans 4294972831 rc 0/0 00000100:00100000:3.0:1789760731.232758:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5543 00000100:00100000:3.0:1789760731.232768:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715489401772 time 715518159702 state 0x2 ref 0 00000100:00100000:1.0:1789760731.237762:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760731.237871:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495785472 00000100:00100000:0.0:1789760731.237881:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495785472 00000100:00100000:0.0:1789760731.237940:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715489401772 time 715518159702 state 0x0 ref 1 00000100:00100000:0.0:1789760731.237958:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715518420118 time 715518159702 state 0x1 ref 1 00000100:00100000:0.0:1789760731.238003:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715518420118 time 715558719375 state 0x0 ref 1 00000100:00100000:0.0:1789760731.238016:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495785472:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760731.238064:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22589440, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb007 00010000:00010000:0.0:1789760731.238074:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545e99 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.238092:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760731.238097:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760731.238104:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545e99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.238134:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545e99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760731.276926:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972832 is committed 00002000:00010000:2.0:1789760731.277078:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22589440 -> 22593536 00010000:00010000:2.0:1789760731.277094:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545e99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.277337:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545e99 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.277347:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545e99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.277355:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760731.277360:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545e99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.277369:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545e99 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.277389:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545e99 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.277402:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545e99 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.277541:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495785472:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39523us (39783us total) trans 4294972832 rc 0/0 00000100:00100000:2.0:1789760731.277552:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5544 00000100:00100000:2.0:1789760731.277563:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715518420118 time 715558719375 state 0x2 ref 0 00000100:00100000:0.0:1789760731.283029:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760731.283115:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495785600 00000100:00100000:2.0:1789760731.283137:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495785600 00000100:00100000:2.0:1789760731.283202:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715518420118 time 715558719375 state 0x0 ref 1 00000100:00100000:2.0:1789760731.283219:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715558979791 time 715558719375 state 0x1 ref 1 00000100:00100000:2.0:1789760731.283259:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715558979791 time 715603975280 state 0x0 ref 1 00000100:00100000:2.0:1789760731.283273:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495785600:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760731.283350:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22593536, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb007 00010000:00010000:2.0:1789760731.283360:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ea0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.283375:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760731.283380:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.283386:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ea0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.283404:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ea0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760731.324583:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22593536 -> 22597632 00010000:00010000:0.0:1789760731.324602:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ea0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.324615:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ea0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.324625:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ea0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.324632:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760731.324638:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ea0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.324648:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ea0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.324667:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ea0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.324676:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545ea0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760731.324775:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495785600:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41499us (41751us total) trans 4294972833 rc 0/0 00000100:00100000:0.0:1789760731.324784:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5545 00000100:00100000:0.0:1789760731.324794:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715558979791 time 715603975280 state 0x2 ref 0 00000001:00080000:0.0:1789760731.324833:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972833 is committed 00000100:00100000:0.0:1789760731.327926:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760731.328010:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495785728 00000100:00100000:3.0:1789760731.328018:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495785728 00000100:00100000:3.0:1789760731.328067:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715558979791 time 715603975280 state 0x0 ref 1 00000100:00100000:3.0:1789760731.328081:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715604235696 time 715603975280 state 0x1 ref 1 00000100:00100000:3.0:1789760731.328168:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715604235696 time 715648883141 state 0x0 ref 1 00000100:00100000:3.0:1789760731.328185:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f921c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495785728:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760731.328327:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22597632, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb007 00010000:00010000:3.0:1789760731.328337:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545ea7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.328353:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760731.328358:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.328364:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545ea7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.328378:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545ea7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760731.361270:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22597632 -> 22601728 00010000:00010000:0.0:1789760731.361322:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545ea7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.361336:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545ea7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.361346:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545ea7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.361353:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760731.361357:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545ea7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.361367:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545ea7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.361386:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545ea7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.361395:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545ea7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760731.361492:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f921c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495785728:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33305us (33567us total) trans 4294972834 rc 0/0 00000100:00100000:0.0:1789760731.361503:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5546 00000100:00100000:0.0:1789760731.361513:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715604235696 time 715648883141 state 0x2 ref 0 00000001:00080000:2.0:1789760731.361779:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972834 is committed 00000100:00100000:1.0:1789760731.366429:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760731.366592:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495785856 00000100:00100000:2.0:1789760731.366601:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495785856 00000100:00100000:2.0:1789760731.366656:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715604235696 time 715648883141 state 0x0 ref 1 00000100:00100000:2.0:1789760731.366671:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715649143557 time 715648883141 state 0x1 ref 1 00000100:00100000:2.0:1789760731.366737:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715649143557 time 715687452202 state 0x0 ref 1 00000100:00100000:2.0:1789760731.366750:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495785856:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760731.366796:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22601728, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb007 00010000:00010000:2.0:1789760731.366804:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545eae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.366819:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760731.366823:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.366830:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545eae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.366843:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545eae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760731.392991:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972835 is committed 00002000:00010000:1.0:1789760731.393113:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22601728 -> 22605824 00010000:00010000:1.0:1789760731.393140:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545eae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.393151:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545eae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.393159:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545eae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.393167:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760731.393171:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545eae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.393180:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545eae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.393199:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545eae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.393207:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354545eae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.393332:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495785856:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26579us (26906us total) trans 4294972835 rc 0/0 00000100:00100000:1.0:1789760731.393342:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5547 00000100:00100000:1.0:1789760731.393354:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715649143557 time 715687452202 state 0x2 ref 0 00000100:00100000:1.0:1789760731.399267:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760731.399642:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495785984 00000100:00100000:0.0:1789760731.399656:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495785984 00000100:00100000:0.0:1789760731.399733:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715649143557 time 715687452202 state 0x0 ref 1 00000100:00100000:0.0:1789760731.399752:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715687712618 time 715687452202 state 0x1 ref 1 00000100:00100000:0.0:1789760731.399786:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715687712618 time 715720502356 state 0x0 ref 1 00000100:00100000:0.0:1789760731.399800:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495785984:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760731.399863:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22605824, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb007 00010000:00010000:0.0:1789760731.399874:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545eb5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.399894:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760731.399900:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760731.399908:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545eb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.399930:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545eb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760731.456378:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972836 is committed 00002000:00010000:1.0:1789760731.456537:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22605824 -> 22609920 00010000:00010000:1.0:1789760731.456553:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545eb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.456567:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545eb5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.456577:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545eb5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.456585:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760731.456590:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545eb5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.456600:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545eb5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.456619:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545eb5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.456627:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545eb5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.456763:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495785984:12345-192.168.203.9@tcp:4:dd.0 Request processed in 56961us (57505us total) trans 4294972836 rc 0/0 00000100:00100000:1.0:1789760731.456773:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5548 00000100:00100000:1.0:1789760731.456781:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715687712618 time 715720502356 state 0x2 ref 0 00000100:00100000:3.0:1789760731.466147:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760731.466765:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495786112 00000100:00100000:0.0:1789760731.466774:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495786112 00000100:00100000:0.0:1789760731.466832:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715687712618 time 715720502356 state 0x0 ref 1 00000100:00100000:0.0:1789760731.466848:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715720762772 time 715720502356 state 0x1 ref 1 00000100:00100000:0.0:1789760731.466883:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715720762772 time 715787599174 state 0x0 ref 1 00000100:00100000:0.0:1789760731.466895:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495786112:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760731.466945:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22609920, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb007 00010000:00010000:0.0:1789760731.466954:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545ebc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.466970:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760731.466975:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760731.466982:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545ebc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.466996:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545ebc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760731.500363:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972837 is committed 00002000:00010000:2.0:1789760731.500559:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22609920 -> 22614016 00010000:00010000:2.0:1789760731.500574:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545ebc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.500587:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545ebc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.500596:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545ebc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.500604:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760731.500609:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545ebc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.500618:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545ebc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.500637:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545ebc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.500644:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354545ebc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.500740:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495786112:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33842us (34597us total) trans 4294972837 rc 0/0 00000100:00100000:2.0:1789760731.500754:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5549 00000100:00100000:2.0:1789760731.500763:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715720762772 time 715787599174 state 0x2 ref 0 00000100:00100000:2.0:1789760731.507535:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760731.507629:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495786240 00000100:00100000:1.0:1789760731.507639:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495786240 00000100:00100000:1.0:1789760731.507694:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715720762772 time 715787599174 state 0x0 ref 1 00000100:00100000:1.0:1789760731.507710:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715787859590 time 715787599174 state 0x1 ref 1 00000100:00100000:1.0:1789760731.507785:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715787859590 time 715828500731 state 0x0 ref 1 00000100:00100000:1.0:1789760731.507799:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495786240:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760731.507851:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22614016, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb007 00010000:00010000:1.0:1789760731.507859:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545ec3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.507874:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760731.507879:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.507885:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545ec3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.507897:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545ec3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760731.542395:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972838 is committed 00002000:00010000:2.0:1789760731.542489:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22614016 -> 22618112 00010000:00010000:2.0:1789760731.542505:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545ec3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.542517:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545ec3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.542527:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545ec3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.542534:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760731.542539:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545ec3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.542547:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545ec3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.542565:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545ec3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.542573:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354545ec3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.542704:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495786240:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34904us (35173us total) trans 4294972838 rc 0/0 00000100:00100000:2.0:1789760731.542714:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5550 00000100:00100000:2.0:1789760731.542725:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715787859590 time 715828500731 state 0x2 ref 0 00000100:00100000:1.0:1789760731.546712:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760731.546847:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495786368 00000100:00100000:0.0:1789760731.546857:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495786368 00000100:00100000:0.0:1789760731.546991:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715787859590 time 715828500731 state 0x0 ref 1 00000100:00100000:0.0:1789760731.547008:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715828761147 time 715828500731 state 0x1 ref 1 00000100:00100000:0.0:1789760731.547073:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715828761147 time 715867788810 state 0x0 ref 1 00000100:00100000:0.0:1789760731.547086:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495786368:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760731.547156:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22618112, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb007 00010000:00010000:0.0:1789760731.547166:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545eca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.547183:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760731.547188:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760731.547195:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545eca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.547210:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545eca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760731.582648:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22618112 -> 22622208 00010000:00010000:2.0:1789760731.582669:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545eca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.582683:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545eca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.582693:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545eca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.582701:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760731.582707:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545eca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.582717:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545eca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.582739:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545eca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.582752:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354545eca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.582857:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495786368:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35769us (36150us total) trans 4294972839 rc 0/0 00000100:00100000:2.0:1789760731.582868:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5551 00000100:00100000:2.0:1789760731.582878:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715828761147 time 715867788810 state 0x2 ref 0 00000001:00080000:3.0:1789760731.583022:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972839 is committed 00000100:00100000:0.0:1789760731.589030:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760731.589124:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495786496 00000100:00100000:2.0:1789760731.589134:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495786496 00000100:00100000:2.0:1789760731.589190:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715828761147 time 715867788810 state 0x0 ref 1 00000100:00100000:2.0:1789760731.589205:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715868049226 time 715867788810 state 0x1 ref 1 00000100:00100000:2.0:1789760731.589240:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715868049226 time 715909956695 state 0x0 ref 1 00000100:00100000:2.0:1789760731.589254:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f923100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495786496:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760731.589346:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22622208, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb007 00010000:00010000:2.0:1789760731.589356:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545ed1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.589372:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760731.589376:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.589383:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545ed1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.589402:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545ed1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760731.622940:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972840 is committed 00002000:00010000:3.0:1789760731.623039:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22622208 -> 22626304 00010000:00010000:3.0:1789760731.623055:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545ed1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.623068:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545ed1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.623082:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545ed1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.623088:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760731.623102:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545ed1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.623111:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545ed1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.623137:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545ed1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.623144:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545ed1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.623270:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f923100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495786496:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34014us (34246us total) trans 4294972840 rc 0/0 00000100:00100000:3.0:1789760731.623316:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5552 00000100:00100000:3.0:1789760731.623326:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715868049226 time 715909956695 state 0x2 ref 0 00000100:00100000:0.0:1789760731.627240:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760731.627353:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495786624 00000100:00100000:2.0:1789760731.627363:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495786624 00000100:00100000:2.0:1789760731.627408:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715868049226 time 715909956695 state 0x0 ref 1 00000100:00100000:2.0:1789760731.627419:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715910217111 time 715909956695 state 0x1 ref 1 00000100:00100000:1.0:1789760731.627481:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715910217111 time 715948195435 state 0x0 ref 1 00000100:00100000:1.0:1789760731.627497:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f922300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495786624:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760731.627546:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22626304, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb007 00010000:00010000:1.0:1789760731.627555:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545ed8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.627571:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760731.627575:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.627581:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545ed8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.627677:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545ed8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760731.655418:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972841 is committed 00002000:00010000:3.0:1789760731.655648:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22626304 -> 22630400 00002000:00010000:3.0:1789760731.655654:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760730 -> 1789760731 00002000:00010000:3.0:1789760731.655657:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760730 -> 1789760731 00010000:00010000:3.0:1789760731.655670:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545ed8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.655685:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545ed8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.655694:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545ed8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.655701:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760731.655706:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545ed8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.655714:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545ed8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.655734:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545ed8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.655742:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354545ed8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.655838:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f922300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495786624:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28340us (28604us total) trans 4294972841 rc 0/0 00000100:00100000:3.0:1789760731.655847:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5553 00000100:00100000:3.0:1789760731.655857:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715910217111 time 715948195435 state 0x2 ref 0 00000100:00100000:1.0:1789760731.660434:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760731.660625:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495786752 00000100:00100000:2.0:1789760731.660635:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495786752 00000100:00100000:2.0:1789760731.660689:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715910217111 time 715948195435 state 0x0 ref 1 00000100:00100000:2.0:1789760731.660704:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715948455851 time 715948195435 state 0x1 ref 1 00000100:00100000:2.0:1789760731.660737:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715948455851 time 715981453754 state 0x0 ref 1 00000100:00100000:2.0:1789760731.660750:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903639e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495786752:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760731.660797:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22630400, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb007 00010000:00010000:2.0:1789760731.660806:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545edf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.660822:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760731.660826:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.660833:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545edf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.660852:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545edf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760731.705947:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972842 is committed 00002000:00010000:0.0:1789760731.706062:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22630400 -> 22634496 00010000:00010000:0.0:1789760731.706078:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545edf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.706106:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545edf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.706127:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545edf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.706134:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760731.706139:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545edf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.706147:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545edf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.706170:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545edf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.706178:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545edf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760731.706272:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903639e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495786752:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45518us (45841us total) trans 4294972842 rc 0/0 00000100:00100000:0.0:1789760731.706324:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5554 00000100:00100000:0.0:1789760731.706335:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715948455851 time 715981453754 state 0x2 ref 0 00000100:00100000:0.0:1789760731.715619:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760731.715693:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495786880 00000100:00100000:3.0:1789760731.715702:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495786880 00000100:00100000:3.0:1789760731.715753:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715948455851 time 715981453754 state 0x0 ref 1 00000100:00100000:3.0:1789760731.715767:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715981714170 time 715981453754 state 0x1 ref 1 00000100:00100000:3.0:1789760731.715799:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715981714170 time 716036515285 state 0x0 ref 1 00000100:00100000:3.0:1789760731.715857:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f921c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495786880:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760731.715897:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22634496, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb007 00010000:00010000:3.0:1789760731.715904:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545ee6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.715916:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760731.715920:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.715924:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545ee6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.715934:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545ee6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760731.749800:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22634496 -> 22638592 00010000:00010000:3.0:1789760731.749820:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545ee6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.749833:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545ee6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.749847:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545ee6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.749854:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760731.749858:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545ee6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.749866:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545ee6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.749884:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545ee6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.749892:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545ee6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.749983:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f921c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495786880:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34123us (34366us total) trans 4294972843 rc 0/0 00000100:00100000:3.0:1789760731.750374:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5555 00000100:00100000:3.0:1789760731.750385:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715981714170 time 716036515285 state 0x2 ref 0 00000001:00080000:0.0:1789760731.750427:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972843 is committed 00000100:00100000:3.0:1789760731.761463:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760731.761590:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495787008 00000100:00100000:2.0:1789760731.761601:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495787008 00000100:00100000:2.0:1789760731.761865:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 715981714170 time 716036515285 state 0x0 ref 1 00000100:00100000:2.0:1789760731.761879:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716036775701 time 716036515285 state 0x1 ref 1 00000100:00100000:2.0:1789760731.761911:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716036775701 time 716082628287 state 0x0 ref 1 00000100:00100000:2.0:1789760731.761924:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495787008:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760731.761971:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22638592, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb007 00010000:00010000:2.0:1789760731.761981:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545eed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.761997:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760731.762001:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.762007:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545eed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.762021:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545eed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760731.807403:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22638592 -> 22642688 00010000:00010000:0.0:1789760731.807423:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545eed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.807437:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545eed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.807446:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545eed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.807454:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760731.807459:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545eed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.807468:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545eed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.807486:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545eed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.807495:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545eed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760731.807584:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495787008:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45658us (46126us total) trans 4294972844 rc 0/0 00000100:00100000:0.0:1789760731.807594:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5556 00000100:00100000:0.0:1789760731.807603:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716036775701 time 716082628287 state 0x2 ref 0 00000001:00080000:3.0:1789760731.808360:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972844 is committed 00000100:00100000:1.0:1789760731.814309:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760731.814340:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495787136 00000100:00100000:0.0:1789760731.814349:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495787136 00000100:00100000:0.0:1789760731.814402:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716036775701 time 716082628287 state 0x0 ref 1 00000100:00100000:0.0:1789760731.814416:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716082888703 time 716082628287 state 0x1 ref 1 00000100:00100000:0.0:1789760731.814447:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716082888703 time 716135163536 state 0x0 ref 1 00000100:00100000:0.0:1789760731.814459:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495787136:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760731.814504:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22642688, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb007 00010000:00010000:0.0:1789760731.814513:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545ef4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.814531:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760731.814535:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760731.814542:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545ef4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.814556:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545ef4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760731.856030:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972845 is committed 00002000:00010000:0.0:1789760731.856154:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22642688 -> 22646784 00010000:00010000:0.0:1789760731.856169:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545ef4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.856198:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545ef4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.856208:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545ef4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.856215:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760731.856220:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545ef4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.856229:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545ef4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.856249:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545ef4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.856257:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545ef4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760731.856393:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495787136:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41931us (42089us total) trans 4294972845 rc 0/0 00000100:00100000:0.0:1789760731.856403:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5557 00000100:00100000:0.0:1789760731.856413:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716082888703 time 716135163536 state 0x2 ref 0 00000100:00100000:1.0:1789760731.860471:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760731.860530:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495787264 00000100:00100000:3.0:1789760731.860540:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495787264 00000100:00100000:3.0:1789760731.860594:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716082888703 time 716135163536 state 0x0 ref 1 00000100:00100000:3.0:1789760731.860610:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716135423952 time 716135163536 state 0x1 ref 1 00000100:00100000:3.0:1789760731.860656:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716135423952 time 716181372240 state 0x0 ref 1 00000100:00100000:3.0:1789760731.860670:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495787264:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760731.860714:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22646784, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb007 00010000:00010000:3.0:1789760731.860724:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545efb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.860739:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760731.860744:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.860751:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545efb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.860764:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545efb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760731.916657:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972846 is committed 00002000:00010000:1.0:1789760731.916781:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22646784 -> 22650880 00010000:00010000:1.0:1789760731.916796:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545efb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.916809:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545efb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.916818:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545efb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.916840:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760731.916845:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545efb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.916854:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545efb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.916874:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545efb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.916882:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545efb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.916971:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495787264:12345-192.168.203.9@tcp:4:dd.0 Request processed in 56300us (56505us total) trans 4294972846 rc 0/0 00000100:00100000:1.0:1789760731.916981:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5558 00000100:00100000:1.0:1789760731.916991:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716135423952 time 716181372240 state 0x2 ref 0 00000100:00100000:3.0:1789760731.922577:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760731.923183:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495787392 00000100:00100000:0.0:1789760731.923193:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495787392 00000100:00100000:0.0:1789760731.923250:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716135423952 time 716181372240 state 0x0 ref 1 00000100:00100000:0.0:1789760731.923267:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716181632656 time 716181372240 state 0x1 ref 1 00000100:00100000:0.0:1789760731.923340:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716181632656 time 716244055568 state 0x0 ref 1 00000100:00100000:0.0:1789760731.923353:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495787392:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760731.923405:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22650880, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb007 00010000:00010000:0.0:1789760731.923414:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354545f02 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.923430:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760731.923833:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760731.923841:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354545f02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.923857:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354545f02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760731.961662:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22650880 -> 22654976 00010000:00010000:0.0:1789760731.961683:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354545f02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.961697:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354545f02 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.961712:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354545f02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.961719:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ae00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760731.961723:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354545f02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.961732:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354545f02 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.961753:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354545f02 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.961760:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354545f02 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760731.961850:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495787392:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38495us (39278us total) trans 4294972847 rc 0/0 00000100:00100000:0.0:1789760731.961861:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5559 00000100:00100000:0.0:1789760731.961870:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716181632656 time 716244055568 state 0x2 ref 0 00000001:00080000:2.0:1789760731.962006:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972847 is committed 00000100:00100000:0.0:1789760731.969047:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760731.969217:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495787520 00000100:00100000:2.0:1789760731.969227:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495787520 00000100:00100000:2.0:1789760731.969310:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716181632656 time 716244055568 state 0x0 ref 1 00000100:00100000:2.0:1789760731.969328:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716244315984 time 716244055568 state 0x1 ref 1 00000100:00100000:2.0:1789760731.969361:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716244315984 time 716290077281 state 0x0 ref 1 00000100:00100000:3.0:1789760731.969652:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f923100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495787520:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760731.969694:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22654976, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb007 00010000:00010000:3.0:1789760731.969701:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545f09 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.969723:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760731.969727:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.969734:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545f09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.969749:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545f09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.998532:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760732.008226:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972848 is committed 00002000:00010000:0.0:1789760732.008347:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22654976 -> 22659072 00010000:00010000:0.0:1789760732.008362:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545f09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.008380:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545f09 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.008399:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545f09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.008407:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760732.008413:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545f09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.008422:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545f09 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.008452:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545f09 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.008461:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545f09 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.008558:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f923100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495787520:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38997us (39516us total) trans 4294972848 rc 0/0 00000100:00100000:0.0:1789760732.008568:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5560 00000100:00100000:0.0:1789760732.008579:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716244315984 time 716290077281 state 0x2 ref 0 00000100:00100000:2.0:1789760732.018917:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760732.019020:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495787648 00000100:00100000:1.0:1789760732.019030:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495787648 00000100:00100000:1.0:1789760732.019084:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716244315984 time 716290077281 state 0x0 ref 1 00000100:00100000:1.0:1789760732.019173:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716290337697 time 716290077281 state 0x1 ref 1 00000100:00100000:1.0:1789760732.019245:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716290337697 time 716339960599 state 0x0 ref 1 00000100:00100000:1.0:1789760732.019259:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495787648:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760732.019336:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22659072, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb007 00010000:00010000:1.0:1789760732.019346:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f10 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.019365:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760732.019370:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.019376:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.019517:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760732.057837:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972849 is committed 00002000:00010000:1.0:1789760732.057937:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22659072 -> 22663168 00010000:00010000:1.0:1789760732.057952:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.057965:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f10 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.057974:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f10 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.057981:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760732.057987:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f10 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.057996:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f10 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.058016:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f10 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.058024:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f10 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.058338:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495787648:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39023us (39372us total) trans 4294972849 rc 0/0 00000100:00100000:1.0:1789760732.058350:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5561 00000100:00100000:1.0:1789760732.058360:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716290337697 time 716339960599 state 0x2 ref 0 00000100:00100000:1.0:1789760732.062193:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760732.062418:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495787776 00000100:00100000:0.0:1789760732.062428:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495787776 00000100:00100000:0.0:1789760732.062552:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716290337697 time 716339960599 state 0x0 ref 1 00000100:00100000:0.0:1789760732.062566:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716340221015 time 716339960599 state 0x1 ref 1 00000100:00100000:0.0:1789760732.062598:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716340221015 time 716383315007 state 0x0 ref 1 00000100:00100000:0.0:1789760732.062610:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495787776:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760732.062657:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22663168, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb007 00010000:00010000:0.0:1789760732.062667:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545f17 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.062682:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760732.062686:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760732.062692:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545f17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.062705:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545f17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760732.107707:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972850 is committed 00002000:00010000:0.0:1789760732.108041:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22663168 -> 22667264 00010000:00010000:0.0:1789760732.108058:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545f17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.108080:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545f17 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.108094:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545f17 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.108102:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760732.108107:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545f17 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.108129:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545f17 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.108148:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545f17 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.108156:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354545f17 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.108333:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495787776:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45720us (46144us total) trans 4294972850 rc 0/0 00000100:00100000:0.0:1789760732.108342:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5562 00000100:00100000:0.0:1789760732.108352:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716340221015 time 716383315007 state 0x2 ref 0 00000100:00100000:0.0:1789760732.118370:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760732.118429:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495787904 00000100:00100000:3.0:1789760732.118439:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495787904 00000100:00100000:3.0:1789760732.118495:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716340221015 time 716383315007 state 0x0 ref 1 00000100:00100000:3.0:1789760732.118622:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716383575423 time 716383315007 state 0x1 ref 1 00000100:00100000:3.0:1789760732.118703:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716383575423 time 716439419108 state 0x0 ref 1 00000100:00100000:3.0:1789760732.118719:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f922300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495787904:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760732.118766:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22667264, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb007 00010000:00010000:3.0:1789760732.118775:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545f1e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.118791:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760732.118795:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.118801:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545f1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.118823:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545f1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760732.165721:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972851 is committed 00002000:00010000:1.0:1789760732.165887:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22667264 -> 22671360 00010000:00010000:1.0:1789760732.165902:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545f1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.165914:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545f1e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.165923:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545f1e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.165935:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760732.165940:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545f1e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.165949:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545f1e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.165969:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545f1e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.165977:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354545f1e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.166120:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f922300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495787904:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47392us (47748us total) trans 4294972851 rc 0/0 00000100:00100000:1.0:1789760732.166130:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5563 00000100:00100000:1.0:1789760732.166140:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716383575423 time 716439419108 state 0x2 ref 0 00000100:00100000:0.0:1789760732.182990:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760732.183895:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495788032 00000100:00100000:1.0:1789760732.183905:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495788032 00000100:00100000:1.0:1789760732.183963:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716383575423 time 716439419108 state 0x0 ref 1 00000100:00100000:1.0:1789760732.183979:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716439679524 time 716439419108 state 0x1 ref 1 00000100:00100000:1.0:1789760732.184065:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716439679524 time 716504780697 state 0x0 ref 1 00000100:00100000:1.0:1789760732.184082:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495788032:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760732.184153:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22671360, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb007 00010000:00010000:1.0:1789760732.184162:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f25 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.184179:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760732.184184:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.184190:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.184204:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760732.233486:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972852 is committed 00002000:00010000:1.0:1789760732.233656:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22671360 -> 22675456 00010000:00010000:1.0:1789760732.233682:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.233696:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f25 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.233705:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.233712:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760732.233717:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.233727:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f25 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.233746:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f25 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.233753:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f25 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.233856:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495788032:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49772us (50871us total) trans 4294972852 rc 0/0 00000100:00100000:1.0:1789760732.233867:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5564 00000100:00100000:1.0:1789760732.233877:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716439679524 time 716504780697 state 0x2 ref 0 00000100:00100000:3.0:1789760732.237583:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760732.239019:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495788160 00000100:00100000:0.0:1789760732.239033:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495788160 00000100:00100000:1.0:1789760732.239445:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716439679524 time 716504780697 state 0x0 ref 1 00000100:00100000:1.0:1789760732.239461:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716505041113 time 716504780697 state 0x1 ref 1 00000100:00100000:1.0:1789760732.239495:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716505041113 time 716560212033 state 0x0 ref 1 00000100:00100000:1.0:1789760732.239508:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495788160:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760732.239554:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22675456, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb007 00010000:00010000:1.0:1789760732.239563:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f2c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.239586:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760732.239590:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.239597:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.239610:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760732.277543:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972853 is committed 00002000:00010000:1.0:1789760732.279367:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22675456 -> 22679552 00010000:00010000:1.0:1789760732.279387:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.279400:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f2c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.279409:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f2c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.279416:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760732.279421:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f2c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.279430:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f2c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.279449:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f2c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.279458:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f2c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.279552:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495788160:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40043us (41979us total) trans 4294972853 rc 0/0 00000100:00100000:1.0:1789760732.279563:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5565 00000100:00100000:1.0:1789760732.279573:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716505041113 time 716560212033 state 0x2 ref 0 00000100:00100000:3.0:1789760732.283883:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760732.284053:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495788288 00000100:00100000:1.0:1789760732.284062:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495788288 00000100:00100000:1.0:1789760732.284230:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716505041113 time 716560212033 state 0x0 ref 1 00000100:00100000:1.0:1789760732.284245:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716560472449 time 716560212033 state 0x1 ref 1 00000100:00100000:1.0:1789760732.284554:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716560472449 time 716605269332 state 0x0 ref 1 00000100:00100000:1.0:1789760732.284569:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495788288:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760732.284627:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22679552, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb007 00010000:00010000:1.0:1789760732.284637:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f33 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.284653:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760732.284658:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.284665:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.284811:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760732.313709:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972854 is committed 00002000:00010000:1.0:1789760732.313875:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22679552 -> 22683648 00010000:00010000:1.0:1789760732.313891:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.313903:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f33 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.313911:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f33 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.313916:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760732.313921:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f33 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.313929:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f33 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.313949:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f33 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.313958:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f33 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.314059:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495788288:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29488us (30180us total) trans 4294972854 rc 0/0 00000100:00100000:1.0:1789760732.314070:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5566 00000100:00100000:1.0:1789760732.314080:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716560472449 time 716605269332 state 0x2 ref 0 00000100:00100000:0.0:1789760732.321925:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760732.323948:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495788416 00000100:00100000:1.0:1789760732.323963:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495788416 00000100:00100000:0.0:1789760732.324256:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716560472449 time 716605269332 state 0x0 ref 1 00000100:00100000:0.0:1789760732.324275:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716605529748 time 716605269332 state 0x1 ref 1 00000100:00100000:0.0:1789760732.324494:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716605529748 time 716645202006 state 0x0 ref 1 00000100:00100000:0.0:1789760732.324511:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495788416:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760732.324567:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22683648, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb007 00010000:00010000:0.0:1789760732.324577:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545f3a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.324592:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760732.324597:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760732.324604:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545f3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.324617:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545f3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760732.361378:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22683648 -> 22687744 00010000:00010000:0.0:1789760732.361398:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545f3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.361411:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545f3a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.361420:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545f3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.361439:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94aa00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760732.361445:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545f3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.361455:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545f3a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.361475:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545f3a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.361483:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545f3a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.361576:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495788416:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37064us (39655us total) trans 4294972855 rc 0/0 00000100:00100000:0.0:1789760732.361587:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5567 00000100:00100000:0.0:1789760732.361606:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716605529748 time 716645202006 state 0x2 ref 0 00000001:00080000:0.0:1789760732.361651:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972855 is committed 00000100:00100000:2.0:1789760732.365061:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760732.365198:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495788544 00000100:00100000:1.0:1789760732.365208:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495788544 00000100:00100000:1.0:1789760732.365261:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716605529748 time 716645202006 state 0x0 ref 1 00000100:00100000:1.0:1789760732.365390:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716645462422 time 716645202006 state 0x1 ref 1 00000100:00100000:1.0:1789760732.365448:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716645462422 time 716686164328 state 0x0 ref 1 00000100:00100000:1.0:1789760732.365461:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495788544:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760732.365510:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22687744, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb007 00010000:00010000:1.0:1789760732.365519:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f41 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.365538:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760732.365543:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.365550:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.365564:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760732.409743:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972856 is committed 00002000:00010000:1.0:1789760732.409929:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22687744 -> 22691840 00010000:00010000:1.0:1789760732.409953:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.409970:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f41 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.409981:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.409990:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760732.409996:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.410007:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f41 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.410033:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f41 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.410041:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f41 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.410767:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495788544:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45297us (45703us total) trans 4294972856 rc 0/0 00000100:00100000:1.0:1789760732.410789:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5568 00000100:00100000:1.0:1789760732.410802:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716645462422 time 716686164328 state 0x2 ref 0 00000100:00100000:2.0:1789760732.418449:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760732.418580:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495788672 00000100:00100000:1.0:1789760732.418595:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495788672 00000100:00100000:1.0:1789760732.418678:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716645462422 time 716686164328 state 0x0 ref 1 00000100:00100000:1.0:1789760732.418698:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716686424744 time 716686164328 state 0x1 ref 1 00000100:00100000:1.0:1789760732.418733:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716686424744 time 716739449672 state 0x0 ref 1 00000100:00100000:1.0:1789760732.418747:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495788672:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760732.418979:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22691840, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb007 00010000:00010000:1.0:1789760732.418991:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f48 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.419012:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760732.419017:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.419091:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.419120:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.449624:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033cea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520305280:0@lo:400:kworker.0 00000100:00100000:1.0:1789760732.449731:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760732.449823:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760732.449832:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520305408:0@lo:400:kworker.0 00000100:00100000:1.0:1789760732.449872:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760732.449919:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520305280 00000100:00100000:1.0:1789760732.449928:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520305280 00000100:00100000:1.0:1789760732.450001:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 472 00000100:00100000:1.0:1789760732.450013:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520305280:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760732.452254:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520305280:12345-0@lo:400:kworker.0 Request processed in 2230us (2510us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760732.452273:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 472 00000100:00080000:1.0:1789760732.452345:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760732.452453:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033cea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520305280:0@lo:400:kworker.0 00000100:00100000:2.0:1789760732.452573:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033ced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3318:1876699520305152:0@lo:400:kworker.0 00000100:00100000:2.0:1789760732.452660:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760732.452715:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3318:1876699520305664:0@lo:400:kworker.0 00000100:00100000:2.0:1789760732.452745:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760732.452786:0:3318:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1789760732.452794:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033cd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:59c4e264-9b75-482b-8813-3d1dc124f384:3318:1876699520305024:0@lo:400:kworker.0 00000100:00100000:2.0:1789760732.452825:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760732.452846:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033cc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3318:1876699520305536:0@lo:400:kworker.0 00000100:00100000:2.0:1789760732.452872:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760732.452904:0:13058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520305408 00000100:00100000:2.0:1789760732.452912:0:13058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520305408 00000100:00100000:2.0:1789760732.452975:0:13058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520305664 00000100:00100000:2.0:1789760732.452978:0:13058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520305664 00000100:00100000:2.0:1789760732.453001:0:13058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 220 00000100:00100000:2.0:1789760732.453010:0:13058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520305408:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760732.453068:0:13058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520305408:12345-0@lo:400:kworker.0 Request processed in 58us (3196us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760732.453075:0:13058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 220 00000100:00100000:2.0:1789760732.453096:0:13058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 221 00000100:00100000:2.0:1789760732.453105:0:13058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3318:x1876699520305664:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760732.453164:0:13058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3318:x1876699520305664:12345-0@lo:400:kworker.0 Request processed in 61us (418us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760732.453170:0:13058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 221 00000100:00100000:2.0:1789760732.453190:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520305152 00000100:00100000:2.0:1789760732.453194:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520305152 00000100:00100000:2.0:1789760732.453219:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520305536 00000100:00100000:2.0:1789760732.453222:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520305536 00000100:00100000:2.0:1789760732.453240:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 473 00000100:00100000:2.0:1789760732.453246:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3318:x1876699520305152:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760732.453312:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3318:x1876699520305152:12345-0@lo:400:kworker.0 Request processed in 65us (652us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760732.453319:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 473 00000100:00100000:2.0:1789760732.453330:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 474 00000100:00100000:2.0:1789760732.453337:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1876699520305536:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760732.453367:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1876699520305536:12345-0@lo:400:kworker.0 Request processed in 32us (495us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760732.453373:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 474 00000100:00100000:2.0:1789760732.453413:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033cc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3318:1876699520305536:0@lo:400:kworker.0 00000100:00100000:2.0:1789760732.453424:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033ced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3318:1876699520305152:0@lo:400:kworker.0 00000100:00100000:2.0:1789760732.453433:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3318:1876699520305664:0@lo:400:kworker.0 00000100:00100000:1.0:1789760732.453731:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520305024 00000100:00100000:1.0:1789760732.453736:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520305024 00000100:00100000:1.0:1789760732.453775:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 397 00000100:00100000:1.0:1789760732.453782:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929012313c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:59c4e264-9b75-482b-8813-3d1dc124f384+19:3318:x1876699520305024:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760732.453832:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929012313c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:59c4e264-9b75-482b-8813-3d1dc124f384+19:3318:x1876699520305024:12345-0@lo:400:kworker.0 Request processed in 50us (1006us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760732.453840:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 397 00000100:00080000:1.0:1789760732.453868:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789760732.453872:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789760732.453874:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760732.453898:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520305408:0@lo:400:kworker.0 00000100:00100000:3.0:1789760732.453964:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033cd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:59c4e264-9b75-482b-8813-3d1dc124f384:3318:1876699520305024:0@lo:400:kworker.0 00000001:00080000:0.0:1789760732.478125:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972857 is committed 00002000:00010000:1.0:1789760732.478383:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22691840 -> 22695936 00010000:00010000:1.0:1789760732.478413:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.478442:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f48 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.478452:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.478460:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760732.478466:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.478476:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f48 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.478502:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f48 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.478511:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f48 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.478645:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495788672:12345-192.168.203.9@tcp:4:dd.0 Request processed in 59892us (60200us total) trans 4294972857 rc 0/0 00000100:00100000:1.0:1789760732.478658:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5569 00000100:00100000:1.0:1789760732.478670:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716686424744 time 716739449672 state 0x2 ref 0 00000100:00100000:2.0:1789760732.484422:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760732.484537:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495788800 00000100:00100000:0.0:1789760732.484550:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495788800 00000100:00100000:0.0:1789760732.484630:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716686424744 time 716739449672 state 0x0 ref 1 00000100:00100000:0.0:1789760732.484649:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716739710088 time 716739449672 state 0x1 ref 1 00000100:00100000:0.0:1789760732.484680:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716739710088 time 716805397244 state 0x0 ref 1 00000100:00100000:0.0:1789760732.484694:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495788800:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760732.484758:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22695936, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb007 00010000:00010000:0.0:1789760732.484769:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545f4f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.484789:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760732.484794:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760732.484803:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545f4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.484819:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545f4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760732.544787:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972858 is committed 00002000:00010000:2.0:1789760732.545652:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22695936 -> 22700032 00010000:00010000:2.0:1789760732.545674:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545f4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.545689:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545f4f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.545699:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545f4f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.545706:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94aa00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760732.545712:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545f4f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.545722:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545f4f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.545744:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545f4f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.545752:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354545f4f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760732.545881:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495788800:12345-192.168.203.9@tcp:4:dd.0 Request processed in 61181us (61466us total) trans 4294972858 rc 0/0 00000100:00100000:2.0:1789760732.545893:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5570 00000100:00100000:2.0:1789760732.545907:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716739710088 time 716805397244 state 0x2 ref 0 00000100:00100000:2.0:1789760732.551467:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760732.553523:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495788928 00000100:00100000:1.0:1789760732.553545:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495788928 00000100:00100000:1.0:1789760732.553637:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716739710088 time 716805397244 state 0x0 ref 1 00000100:00100000:1.0:1789760732.553657:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716805657660 time 716805397244 state 0x1 ref 1 00000100:00100000:1.0:1789760732.553696:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716805657660 time 716874412198 state 0x0 ref 1 00000100:00100000:1.0:1789760732.553710:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04292a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495788928:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760732.553781:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22700032, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb007 00010000:00010000:1.0:1789760732.553793:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f56 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.553815:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760732.553820:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.554234:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.554254:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760732.600733:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972859 is committed 00002000:00010000:1.0:1789760732.600828:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22700032 -> 22704128 00010000:00010000:1.0:1789760732.600849:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.600866:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f56 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.600877:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f56 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.600885:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760732.600891:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f56 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.600901:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f56 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.600928:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f56 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.600937:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f56 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.601063:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04292a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495788928:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47349us (49603us total) trans 4294972859 rc 0/0 00000100:00100000:1.0:1789760732.601075:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5571 00000100:00100000:1.0:1789760732.601482:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716805657660 time 716874412198 state 0x2 ref 0 00000100:00100000:3.0:1789760732.608911:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760732.609037:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495789056 00000100:00100000:1.0:1789760732.609051:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495789056 00000100:00100000:1.0:1789760732.609136:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716805657660 time 716874412198 state 0x0 ref 1 00000100:00100000:1.0:1789760732.609155:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716874672614 time 716874412198 state 0x1 ref 1 00000100:00100000:1.0:1789760732.609199:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716874672614 time 716929914719 state 0x0 ref 1 00000100:00100000:1.0:1789760732.609213:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495789056:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760732.609325:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22704128, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb007 00010000:00010000:1.0:1789760732.609336:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f5d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.609357:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760732.609362:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.609369:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.609385:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760732.694800:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22704128 -> 22708224 00010000:00010000:2.0:1789760732.694833:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.694850:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f5d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.694860:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f5d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.694869:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760732.694875:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f5d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.694885:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f5d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.694915:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f5d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.694924:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354545f5d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760732.695069:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495789056:12345-192.168.203.9@tcp:4:dd.0 Request processed in 85850us (86165us total) trans 4294972860 rc 0/0 00000100:00100000:2.0:1789760732.695084:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5572 00000100:00100000:2.0:1789760732.695447:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716874672614 time 716929914719 state 0x2 ref 0 00000001:00080000:3.0:1789760732.696205:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972860 is committed 00000100:00100000:0.0:1789760732.707275:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760732.711019:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495789184 00000100:00100000:2.0:1789760732.711043:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495789184 00000100:00100000:3.0:1789760732.712524:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716874672614 time 716929914719 state 0x0 ref 1 00000100:00100000:3.0:1789760732.712548:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716930175135 time 716929914719 state 0x1 ref 1 00000100:00100000:3.0:1789760732.712598:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716930175135 time 717033314027 state 0x0 ref 1 00000100:00100000:3.0:1789760732.712616:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495789184:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760732.712963:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22708224, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb007 00010000:00010000:3.0:1789760732.712975:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545f64 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.712998:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760732.713003:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.713011:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545f64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.713034:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545f64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760732.765618:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972861 is committed 00002000:00010000:3.0:1789760732.765744:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22708224 -> 22712320 00002000:00010000:3.0:1789760732.765750:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760731 -> 1789760732 00002000:00010000:3.0:1789760732.765753:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760731 -> 1789760732 00010000:00010000:3.0:1789760732.765771:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545f64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.765788:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545f64 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.765799:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545f64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.765807:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760732.765813:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545f64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.765823:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545f64 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.765848:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545f64 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.765861:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545f64 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.766030:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495789184:12345-192.168.203.9@tcp:4:dd.0 Request processed in 53412us (58762us total) trans 4294972861 rc 0/0 00000100:00100000:3.0:1789760732.766043:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5573 00000100:00100000:3.0:1789760732.766056:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716930175135 time 717033314027 state 0x2 ref 0 00000100:00100000:0.0:1789760732.770611:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760732.770715:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495789312 00000100:00100000:1.0:1789760732.770727:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495789312 00000100:00100000:1.0:1789760732.770793:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 716930175135 time 717033314027 state 0x0 ref 1 00000100:00100000:1.0:1789760732.770811:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717033574443 time 717033314027 state 0x1 ref 1 00000100:00100000:1.0:1789760732.770842:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717033574443 time 717091559071 state 0x0 ref 1 00000100:00100000:1.0:1789760732.770856:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03095500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495789312:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760732.770917:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22712320, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb007 00010000:00010000:1.0:1789760732.770928:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f6b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.770948:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760732.770953:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.770961:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.770977:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760732.803177:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22712320 -> 22716416 00010000:00010000:2.0:1789760732.803206:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.803222:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f6b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.803233:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.803241:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760732.803247:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.803258:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f6b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.803305:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f6b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.803314:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354545f6b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760732.803447:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495789312:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32585us (32842us total) trans 4294972862 rc 0/0 00000100:00100000:2.0:1789760732.803460:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5574 00000100:00100000:2.0:1789760732.803473:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717033574443 time 717091559071 state 0x2 ref 0 00000001:00080000:3.0:1789760732.803656:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972862 is committed 00000100:00100000:2.0:1789760732.811850:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760732.812997:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495789440 00000100:00100000:1.0:1789760732.813016:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495789440 00000100:00100000:1.0:1789760732.813104:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717033574443 time 717091559071 state 0x0 ref 1 00000100:00100000:1.0:1789760732.813132:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717091819487 time 717091559071 state 0x1 ref 1 00000100:00100000:1.0:1789760732.813172:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717091819487 time 717133888312 state 0x0 ref 1 00000100:00100000:1.0:1789760732.813188:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04290700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495789440:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760732.813255:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22716416, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb007 00010000:00010000:1.0:1789760732.813267:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545f72 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.813328:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760732.813334:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.813342:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545f72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.813359:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545f72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760732.843379:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22716416 -> 22720512 00010000:00010000:3.0:1789760732.843405:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545f72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.843422:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545f72 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.843432:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545f72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.843441:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760732.843448:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545f72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.843459:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545f72 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.843491:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545f72 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.843500:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354545f72 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.843623:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04290700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495789440:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30431us (31779us total) trans 4294972863 rc 0/0 00000100:00100000:3.0:1789760732.843637:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5575 00000100:00100000:3.0:1789760732.843650:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717091819487 time 717133888312 state 0x2 ref 0 00000001:00080000:0.0:1789760732.843819:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972863 is committed 00000100:00100000:1.0:1789760732.849701:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760732.849801:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495789568 00000100:00100000:3.0:1789760732.849816:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495789568 00000100:00100000:3.0:1789760732.849905:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717091819487 time 717133888312 state 0x0 ref 1 00000100:00100000:3.0:1789760732.849926:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717134148728 time 717133888312 state 0x1 ref 1 00000100:00100000:3.0:1789760732.850019:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717134148728 time 717170733471 state 0x0 ref 1 00000100:00100000:3.0:1789760732.850034:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495789568:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760732.850117:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22720512, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb007 00010000:00010000:3.0:1789760732.850130:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545f79 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.850151:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760732.850157:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.850165:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545f79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.850180:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545f79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760732.872702:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22720512 -> 22724608 00010000:00010000:0.0:1789760732.872730:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545f79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.872751:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545f79 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.872761:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545f79 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.872768:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760732.872773:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545f79 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.872784:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545f79 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.872809:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545f79 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.872817:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354545f79 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.872990:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495789568:12345-192.168.203.9@tcp:4:dd.0 Request processed in 22951us (23298us total) trans 4294972864 rc 0/0 00000100:00100000:0.0:1789760732.873005:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5576 00000100:00100000:0.0:1789760732.873019:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717134148728 time 717170733471 state 0x2 ref 0 00000001:00080000:1.0:1789760732.873145:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972864 is committed 00000100:00100000:1.0:1789760732.877685:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760732.877806:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495789696 00000100:00100000:2.0:1789760732.877820:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495789696 00000100:00100000:2.0:1789760732.877900:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717134148728 time 717170733471 state 0x0 ref 1 00000100:00100000:2.0:1789760732.877919:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717170993887 time 717170733471 state 0x1 ref 1 00000100:00100000:2.0:1789760732.877963:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717170993887 time 717198678714 state 0x0 ref 1 00000100:00100000:2.0:1789760732.877979:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495789696:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760732.878046:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22724608, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb007 00010000:00010000:2.0:1789760732.878057:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545f80 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.878077:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760732.878082:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760732.878090:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545f80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.878115:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545f80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760732.927469:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22724608 -> 22728704 00010000:00010000:1.0:1789760732.927503:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545f80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.927522:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545f80 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.927533:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545f80 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.927543:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760732.927549:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545f80 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.927561:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545f80 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.927589:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545f80 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.927605:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354545f80 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.927897:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495789696:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49914us (50220us total) trans 4294972865 rc 0/0 00000100:00100000:1.0:1789760732.927910:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5577 00000100:00100000:1.0:1789760732.927922:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717170993887 time 717198678714 state 0x2 ref 0 00000001:00080000:0.0:1789760732.929052:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972865 is committed 00000100:00100000:3.0:1789760732.933019:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760732.937835:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495789824 00000100:00100000:1.0:1789760732.937861:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495789824 00000100:00100000:1.0:1789760732.937959:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717170993887 time 717198678714 state 0x0 ref 1 00000100:00100000:1.0:1789760732.937980:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717198939130 time 717198678714 state 0x1 ref 1 00000100:00100000:1.0:1789760732.938012:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717198939130 time 717258729164 state 0x0 ref 1 00000100:00100000:1.0:1789760732.938029:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495789824:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760732.938101:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22728704, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb007 00010000:00010000:1.0:1789760732.938123:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545f87 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.938145:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760732.938151:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.938159:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545f87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.938181:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545f87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.958703:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520305792:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760732.958803:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760732.958928:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520305792 00000100:00100000:0.0:1789760732.958939:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520305792 00000100:00100000:0.0:1789760732.959006:0:6711:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 196 00000100:00100000:0.0:1789760732.959017:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520305792:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789760732.959063:0:6711:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760732.971250:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520305792:12345-0@lo:13:osp-pre-1-0.0 Request processed in 12218us (12432us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760732.971270:0:6711:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 196 00000100:00100000:1.0:1789760732.974392:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520305792:0@lo:13:osp-pre-1-0.0 00000001:00080000:1.0:1789760732.997810:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972866 is committed 00002000:00010000:0.0:1789760732.998008:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22728704 -> 22732800 00010000:00010000:0.0:1789760732.998029:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545f87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.998044:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545f87 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.998056:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545f87 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.998065:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760732.998070:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545f87 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.998081:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545f87 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.998235:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545f87 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.998247:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354545f87 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.998498:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495789824:12345-192.168.203.9@tcp:4:dd.0 Request processed in 60466us (65487us total) trans 4294972866 rc 0/0 00000100:00100000:0.0:1789760732.998511:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5578 00000100:00100000:0.0:1789760732.998525:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717198939130 time 717258729164 state 0x2 ref 0 00000100:00100000:3.0:1789760733.002783:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760733.003235:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495789952 00000100:00100000:0.0:1789760733.003248:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495789952 00000100:00100000:0.0:1789760733.003387:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717198939130 time 717258729164 state 0x0 ref 1 00000100:00100000:0.0:1789760733.003408:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717258989580 time 717258729164 state 0x1 ref 1 00000100:00100000:3.0:1789760733.003471:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717258989580 time 717324183110 state 0x0 ref 1 00000100:00100000:3.0:1789760733.003487:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495789952:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760733.003556:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22732800, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb007 00010000:00010000:3.0:1789760733.003568:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545f8e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.003589:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760733.003594:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.003601:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545f8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.003616:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545f8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760733.004329:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1789760733.025837:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760733.048059:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972867 is committed 00002000:00010000:2.0:1789760733.048265:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22732800 -> 22736896 00010000:00010000:2.0:1789760733.048605:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545f8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.048621:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545f8e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.048631:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545f8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.048640:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760733.048646:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545f8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.048656:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545f8e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.048683:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545f8e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.048692:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354545f8e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.048823:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495789952:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45334us (46049us total) trans 4294972867 rc 0/0 00000100:00100000:2.0:1789760733.048840:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5579 00000100:00100000:2.0:1789760733.048853:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717258989580 time 717324183110 state 0x2 ref 0 00000100:00100000:1.0:1789760733.052386:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760733.052716:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495790080 00000100:00100000:2.0:1789760733.052732:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495790080 00000100:00100000:2.0:1789760733.052815:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717258989580 time 717324183110 state 0x0 ref 1 00000100:00100000:2.0:1789760733.052836:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717324443526 time 717324183110 state 0x1 ref 1 00000100:00100000:2.0:1789760733.052916:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717324443526 time 717373631659 state 0x0 ref 1 00000100:00100000:2.0:1789760733.052930:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495790080:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760733.052999:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22736896, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb007 00010000:00010000:2.0:1789760733.053010:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545f95 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.053030:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760733.053035:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.053043:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545f95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.053061:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545f95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760733.146191:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22736896 -> 22740992 00010000:00010000:1.0:1789760733.146220:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545f95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.146238:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545f95 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.146249:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545f95 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.146258:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760733.146264:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545f95 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.146274:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545f95 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.146345:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545f95 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.146354:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354545f95 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760733.146473:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495790080:12345-192.168.203.9@tcp:4:dd.0 Request processed in 93537us (94195us total) trans 4294972868 rc 0/0 00000100:00100000:1.0:1789760733.146487:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5580 00000100:00100000:1.0:1789760733.146501:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717324443526 time 717373631659 state 0x2 ref 0 00000001:00080000:1.0:1789760733.146570:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972868 is committed 00000100:00100000:3.0:1789760733.150559:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760733.151651:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495790208 00000100:00100000:0.0:1789760733.151736:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495790208 00000100:00100000:1.0:1789760733.151872:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717324443526 time 717373631659 state 0x0 ref 1 00000100:00100000:1.0:1789760733.151891:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717373892075 time 717373631659 state 0x1 ref 1 00000100:00100000:1.0:1789760733.151932:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717373892075 time 717472648420 state 0x0 ref 1 00000100:00100000:1.0:1789760733.151948:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495790208:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760733.152030:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22740992, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb007 00010000:00010000:1.0:1789760733.152042:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354545f9c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.152065:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760733.152071:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760733.152080:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354545f9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.152098:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354545f9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760733.207371:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22740992 -> 22745088 00010000:00010000:2.0:1789760733.207403:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354545f9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.207418:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354545f9c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.207440:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354545f9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.207449:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760733.207454:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354545f9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.207465:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354545f9c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.207493:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354545f9c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.207501:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354545f9c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.207633:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495790208:12345-192.168.203.9@tcp:4:dd.0 Request processed in 55682us (57145us total) trans 4294972869 rc 0/0 00000100:00100000:2.0:1789760733.207646:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5581 00000100:00100000:2.0:1789760733.207659:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717373892075 time 717472648420 state 0x2 ref 0 00000001:00080000:3.0:1789760733.207829:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972869 is committed 00000100:00100000:3.0:1789760733.211167:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760733.211351:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495790336 00000100:00100000:2.0:1789760733.211365:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495790336 00000100:00100000:2.0:1789760733.211450:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717373892075 time 717472648420 state 0x0 ref 1 00000100:00100000:2.0:1789760733.211471:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717472908836 time 717472648420 state 0x1 ref 1 00000100:00100000:2.0:1789760733.211502:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717472908836 time 717532219288 state 0x0 ref 1 00000100:00100000:2.0:1789760733.211516:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495790336:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760733.211580:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22745088, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb007 00010000:00010000:2.0:1789760733.211591:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fa3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.211612:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760733.211618:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.211625:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fa3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.211642:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fa3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760733.244220:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972870 is committed 00002000:00010000:2.0:1789760733.244354:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22745088 -> 22749184 00010000:00010000:2.0:1789760733.244375:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fa3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.244393:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fa3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.244410:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fa3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.244418:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760733.244434:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fa3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.244446:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fa3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.244474:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fa3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.244484:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fa3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.244606:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495790336:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33084us (33445us total) trans 4294972870 rc 0/0 00000100:00100000:2.0:1789760733.244618:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5582 00000100:00100000:2.0:1789760733.244631:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717472908836 time 717532219288 state 0x2 ref 0 00000100:00100000:3.0:1789760733.247863:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760733.248165:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495790464 00000100:00100000:2.0:1789760733.248180:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495790464 00000100:00100000:2.0:1789760733.248257:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717472908836 time 717532219288 state 0x0 ref 1 00000100:00100000:2.0:1789760733.248675:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717532479704 time 717532219288 state 0x1 ref 1 00000100:00100000:2.0:1789760733.248822:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717532479704 time 717569536136 state 0x0 ref 1 00000100:00100000:2.0:1789760733.248839:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495790464:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760733.248907:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22749184, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb007 00010000:00010000:2.0:1789760733.248918:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545faa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.248938:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760733.248943:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.248951:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545faa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.248968:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545faa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760733.284312:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972871 is committed 00002000:00010000:2.0:1789760733.284528:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22749184 -> 22753280 00010000:00010000:2.0:1789760733.284550:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545faa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.284566:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545faa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.284577:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545faa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.284585:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760733.284592:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545faa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.284603:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545faa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.284627:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545faa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.284634:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545faa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.284781:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495790464:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35939us (36924us total) trans 4294972871 rc 0/0 00000100:00100000:2.0:1789760733.284793:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5583 00000100:00100000:2.0:1789760733.284807:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717532479704 time 717569536136 state 0x2 ref 0 00000100:00100000:3.0:1789760733.288151:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760733.288239:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495790592 00000100:00100000:1.0:1789760733.288253:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495790592 00000100:00100000:1.0:1789760733.288356:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717532479704 time 717569536136 state 0x0 ref 1 00000100:00100000:1.0:1789760733.288375:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717569796552 time 717569536136 state 0x1 ref 1 00000100:00100000:1.0:1789760733.288412:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717569796552 time 717609129574 state 0x0 ref 1 00000100:00100000:1.0:1789760733.288433:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495790592:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760733.288499:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22753280, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb007 00010000:00010000:1.0:1789760733.288510:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354545fb1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.288532:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760733.288537:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760733.288545:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354545fb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.288562:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354545fb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760733.321129:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22753280 -> 22757376 00010000:00010000:1.0:1789760733.321157:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354545fb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.321174:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354545fb1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.321184:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354545fb1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.321192:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760733.321198:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354545fb1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.321209:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354545fb1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.321233:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354545fb1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.321242:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354545fb1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760733.321398:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495790592:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32968us (33326us total) trans 4294972872 rc 0/0 00000100:00100000:1.0:1789760733.321415:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5584 00000100:00100000:1.0:1789760733.321683:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717569796552 time 717609129574 state 0x2 ref 0 00000001:00080000:1.0:1789760733.323421:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972872 is committed 00000100:00100000:2.0:1789760733.324822:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760733.324946:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495790720 00000100:00100000:3.0:1789760733.324960:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495790720 00000100:00100000:3.0:1789760733.325033:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717569796552 time 717609129574 state 0x0 ref 1 00000100:00100000:3.0:1789760733.325051:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717609389990 time 717609129574 state 0x1 ref 1 00000100:00100000:3.0:1789760733.325087:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717609389990 time 717645798240 state 0x0 ref 1 00000100:00100000:3.0:1789760733.325110:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495790720:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760733.325177:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22757376, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb007 00010000:00010000:3.0:1789760733.325189:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545fb8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.325210:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760733.325215:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.325222:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545fb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.325240:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545fb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760733.376821:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972873 is committed 00002000:00010000:0.0:1789760733.376972:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22757376 -> 22761472 00010000:00010000:0.0:1789760733.376993:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545fb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.377010:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545fb8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.377026:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545fb8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.377039:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760733.377045:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545fb8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.377055:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545fb8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.377078:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545fb8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.377087:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354545fb8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760733.377226:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495790720:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52119us (52408us total) trans 4294972873 rc 0/0 00000100:00100000:0.0:1789760733.377238:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5585 00000100:00100000:0.0:1789760733.377251:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717609389990 time 717645798240 state 0x2 ref 0 00000100:00100000:1.0:1789760733.386507:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760733.390753:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495790848 00000100:00100000:2.0:1789760733.390777:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495790848 00000100:00100000:2.0:1789760733.390873:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717609389990 time 717645798240 state 0x0 ref 1 00000100:00100000:2.0:1789760733.390895:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717646058656 time 717645798240 state 0x1 ref 1 00000100:00100000:2.0:1789760733.391002:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717646058656 time 717711717722 state 0x0 ref 1 00000100:00100000:2.0:1789760733.391019:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495790848:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760733.391108:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22761472, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb007 00010000:00010000:2.0:1789760733.391121:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545fbf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.391142:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760733.391147:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.391154:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545fbf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.391172:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545fbf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760733.430384:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22761472 -> 22765568 00010000:00010000:2.0:1789760733.430412:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545fbf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.430427:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545fbf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.430437:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545fbf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.430445:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760733.430450:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545fbf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.430460:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545fbf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.430487:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545fbf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.430495:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354545fbf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760733.430751:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972874 is committed 00000100:00100000:2.0:1789760733.430814:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495790848:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39791us (44313us total) trans 4294972874 rc 0/0 00000100:00100000:2.0:1789760733.430828:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5586 00000100:00100000:2.0:1789760733.430841:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717646058656 time 717711717722 state 0x2 ref 0 00000100:00100000:1.0:1789760733.434749:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760733.434854:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495790976 00000100:00100000:2.0:1789760733.434867:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495790976 00000100:00100000:2.0:1789760733.434933:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717646058656 time 717711717722 state 0x0 ref 1 00000100:00100000:2.0:1789760733.434952:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717711978138 time 717711717722 state 0x1 ref 1 00000100:00100000:2.0:1789760733.435127:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717711978138 time 717755841440 state 0x0 ref 1 00000100:00100000:2.0:1789760733.435142:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495790976:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760733.435209:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22765568, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb007 00010000:00010000:2.0:1789760733.435220:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fc6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.435238:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760733.435243:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.435251:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fc6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.435266:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fc6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760733.485925:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972875 is committed 00002000:00010000:2.0:1789760733.486054:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22765568 -> 22769664 00010000:00010000:2.0:1789760733.486074:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fc6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.486110:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fc6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.486122:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fc6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.486131:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760733.486137:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fc6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.486148:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fc6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.486173:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fc6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.486181:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fc6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.486504:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495790976:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51358us (51762us total) trans 4294972875 rc 0/0 00000100:00100000:2.0:1789760733.486518:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5587 00000100:00100000:2.0:1789760733.486531:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717711978138 time 717755841440 state 0x2 ref 0 00000100:00100000:2.0:1789760733.489765:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760733.489895:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495791104 00000100:00100000:0.0:1789760733.489909:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495791104 00000100:00100000:0.0:1789760733.489986:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717711978138 time 717755841440 state 0x0 ref 1 00000100:00100000:0.0:1789760733.490006:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717756101856 time 717755841440 state 0x1 ref 1 00000100:00100000:0.0:1789760733.490113:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717756101856 time 717810819823 state 0x0 ref 1 00000100:00100000:0.0:1789760733.490130:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04291180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495791104:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760733.490197:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22769664, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb007 00010000:00010000:0.0:1789760733.490208:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fcd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.490229:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760733.490234:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760733.490242:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fcd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.490258:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fcd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760733.527756:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972876 is committed 00002000:00010000:0.0:1789760733.528186:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22769664 -> 22773760 00010000:00010000:0.0:1789760733.528206:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fcd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.528222:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fcd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.528232:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fcd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.528241:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760733.528247:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fcd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.528259:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fcd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.528324:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fcd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.528334:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fcd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760733.528450:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04291180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495791104:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38316us (38692us total) trans 4294972876 rc 0/0 00000100:00100000:0.0:1789760733.528463:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5588 00000100:00100000:0.0:1789760733.528476:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717756101856 time 717810819823 state 0x2 ref 0 00000100:00100000:1.0:1789760733.531998:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760733.532658:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495791232 00000100:00100000:2.0:1789760733.532672:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495791232 00000100:00100000:2.0:1789760733.532747:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717756101856 time 717810819823 state 0x0 ref 1 00000100:00100000:2.0:1789760733.532766:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717811080239 time 717810819823 state 0x1 ref 1 00000100:00100000:2.0:1789760733.532807:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717811080239 time 717853522599 state 0x0 ref 1 00000100:00100000:2.0:1789760733.532822:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495791232:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760733.532892:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22773760, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb007 00010000:00010000:2.0:1789760733.532903:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fd4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.532923:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760733.532929:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.532936:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fd4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.532953:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fd4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760733.595176:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972877 is committed 00002000:00010000:0.0:1789760733.595347:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22773760 -> 22777856 00010000:00010000:0.0:1789760733.595382:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fd4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.595399:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fd4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.595410:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fd4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.595419:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760733.595426:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fd4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.595436:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fd4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.595464:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fd4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.595473:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354545fd4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760733.595600:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495791232:12345-192.168.203.9@tcp:4:dd.0 Request processed in 62773us (63609us total) trans 4294972877 rc 0/0 00000100:00100000:0.0:1789760733.595613:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5589 00000100:00100000:0.0:1789760733.595627:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717811080239 time 717853522599 state 0x2 ref 0 00000100:00100000:1.0:1789760733.602257:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760733.602728:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495791360 00000100:00100000:2.0:1789760733.602807:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495791360 00000100:00100000:2.0:1789760733.602886:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717811080239 time 717853522599 state 0x0 ref 1 00000100:00100000:2.0:1789760733.602906:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717853783015 time 717853522599 state 0x1 ref 1 00000100:00100000:2.0:1789760733.602981:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717853783015 time 717923697020 state 0x0 ref 1 00000100:00100000:2.0:1789760733.602996:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495791360:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760733.603064:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22777856, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb007 00010000:00010000:2.0:1789760733.603076:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545fdb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.603179:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760733.603184:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.603193:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545fdb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.603210:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545fdb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760733.646755:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972878 is committed 00002000:00010000:3.0:1789760733.646984:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22777856 -> 22781952 00010000:00010000:3.0:1789760733.647004:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545fdb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.647020:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545fdb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.647030:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545fdb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.647038:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760733.647044:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545fdb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.647055:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545fdb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.647090:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545fdb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.647107:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354545fdb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.647272:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495791360:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44268us (45020us total) trans 4294972878 rc 0/0 00000100:00100000:3.0:1789760733.647339:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5590 00000100:00100000:3.0:1789760733.647645:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717853783015 time 717923697020 state 0x2 ref 0 00000100:00100000:2.0:1789760733.655762:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760733.655888:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495791488 00000100:00100000:0.0:1789760733.655902:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495791488 00000100:00100000:0.0:1789760733.655980:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717853783015 time 717923697020 state 0x0 ref 1 00000100:00100000:0.0:1789760733.656001:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717923957436 time 717923697020 state 0x1 ref 1 00000100:00100000:0.0:1789760733.656038:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717923957436 time 717976754073 state 0x0 ref 1 00000100:00100000:0.0:1789760733.656052:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495791488:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760733.656130:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22781952, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb007 00010000:00010000:0.0:1789760733.656143:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fe2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.656164:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760733.656169:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760733.656176:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fe2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.656191:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fe2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760733.685841:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:2.0:1789760733.689586:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972879 is committed 00000100:00100000:0.0:1789760733.689950:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760733.690015:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00002000:00010000:0.0:1789760733.690096:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22781952 -> 22786048 00002000:00010000:0.0:1789760733.690109:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760732 -> 1789760733 00002000:00010000:0.0:1789760733.690112:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760732 -> 1789760733 00010000:00010000:0.0:1789760733.690129:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fe2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.690144:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fe2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.690155:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fe2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.690163:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760733.690170:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fe2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.690180:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fe2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.690207:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fe2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.690214:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fe2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760733.690336:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495791488:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34284us (34588us total) trans 4294972879 rc 0/0 00000100:00100000:0.0:1789760733.690348:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5591 00000100:00100000:0.0:1789760733.690362:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717923957436 time 717976754073 state 0x2 ref 0 00000100:00100000:3.0:1789760733.690971:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495791872 00000100:00100000:3.0:1789760733.690984:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495791872 00000100:00100000:3.0:1789760733.691066:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495791744 00000100:00100000:3.0:1789760733.691069:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495791744 00000100:00100000:3.0:1789760733.691115:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 475 00000100:00100000:3.0:1789760733.691125:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03097b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2359:x1876699495791872:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:3.0:1789760733.691197:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03097b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2359:x1876699495791872:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 73us (1254us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760733.691203:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 475 00000100:00100000:3.0:1789760733.691219:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 476 00000100:00100000:3.0:1789760733.691226:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03094700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:17022965-1e4f-46fb-915c-61730096b8e2+8:2359:x1876699495791744:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:1.0:1789760733.691352:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495791616 00000100:00100000:1.0:1789760733.691358:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495791616 00000100:00100000:1.0:1789760733.691464:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 398 00000100:00100000:1.0:1789760733.691473:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929012312c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2359:x1876699495791616:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:1.0:1789760733.691506:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929012312c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2359:x1876699495791616:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 35us (5682us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760733.691512:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 398 00000100:00080000:0.0:1789760733.691629:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760733.691700:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03094700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:17022965-1e4f-46fb-915c-61730096b8e2+8:2359:x1876699495791744:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 475us (1685us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760733.691708:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 476 00000100:00100000:1.0:1789760733.696264:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760733.696488:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495792000 00000100:00100000:0.0:1789760733.696503:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495792000 00000100:00100000:0.0:1789760733.696579:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717923957436 time 717976754073 state 0x0 ref 1 00000100:00100000:0.0:1789760733.696600:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717977014489 time 717976754073 state 0x1 ref 1 00000100:00100000:0.0:1789760733.696626:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717977014489 time 718017343047 state 0x0 ref 1 00000100:00100000:0.0:1789760733.696640:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495792000:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760733.696718:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22786048, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb007 00010000:00010000:0.0:1789760733.696731:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fe9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.696751:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760733.696756:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760733.696764:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fe9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.696780:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fe9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760733.759400:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972880 is committed 00002000:00010000:0.0:1789760733.759578:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22786048 -> 22790144 00010000:00010000:0.0:1789760733.759599:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fe9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.759615:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fe9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.759626:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fe9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.759634:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760733.759640:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fe9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.759651:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fe9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.759675:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fe9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.759746:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545fe9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760733.759940:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495792000:12345-192.168.203.9@tcp:4:dd.0 Request processed in 63294us (63683us total) trans 4294972880 rc 0/0 00000100:00100000:0.0:1789760733.759952:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5592 00000100:00100000:0.0:1789760733.759965:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717977014489 time 718017343047 state 0x2 ref 0 00000100:00100000:3.0:1789760733.765961:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760733.766530:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495792128 00000100:00100000:2.0:1789760733.766546:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495792128 00000100:00100000:2.0:1789760733.766628:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 717977014489 time 718017343047 state 0x0 ref 1 00000100:00100000:2.0:1789760733.766648:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718017603463 time 718017343047 state 0x1 ref 1 00000100:00100000:2.0:1789760733.766695:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718017603463 time 718087402934 state 0x0 ref 1 00000100:00100000:2.0:1789760733.766711:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495792128:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760733.766778:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22790144, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb007 00010000:00010000:2.0:1789760733.766791:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545ff0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.766811:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760733.766816:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.766824:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545ff0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.766840:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545ff0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760733.799890:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972881 is committed 00002000:00010000:2.0:1789760733.800007:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22790144 -> 22794240 00010000:00010000:2.0:1789760733.800028:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545ff0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.800042:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545ff0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.800053:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545ff0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.800061:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760733.800066:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545ff0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.800141:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545ff0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.800216:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545ff0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.800224:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354545ff0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.800371:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495792128:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33657us (34424us total) trans 4294972881 rc 0/0 00000100:00100000:2.0:1789760733.800383:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5593 00000100:00100000:2.0:1789760733.800397:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718017603463 time 718087402934 state 0x2 ref 0 00000100:00100000:1.0:1789760733.805005:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760733.805183:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495792256 00000100:00100000:0.0:1789760733.805195:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495792256 00000100:00100000:0.0:1789760733.809681:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718017603463 time 718087402934 state 0x0 ref 1 00000100:00100000:0.0:1789760733.809747:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718087663350 time 718087402934 state 0x1 ref 1 00000100:00100000:0.0:1789760733.809807:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718087663350 time 718130523561 state 0x0 ref 1 00000100:00100000:0.0:1789760733.809839:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495792256:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760733.809915:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22794240, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb007 00010000:00010000:0.0:1789760733.809928:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545ff7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.809954:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760733.809959:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760733.809967:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545ff7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.809982:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545ff7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760733.863320:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972882 is committed 00002000:00010000:2.0:1789760733.863486:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22794240 -> 22798336 00010000:00010000:2.0:1789760733.863508:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545ff7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.863531:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545ff7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.863540:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545ff7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.863548:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760733.863555:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545ff7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.863566:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545ff7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.863591:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545ff7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.863600:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354545ff7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.863727:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495792256:12345-192.168.203.9@tcp:4:dd.0 Request processed in 53900us (58730us total) trans 4294972882 rc 0/0 00000100:00100000:2.0:1789760733.863741:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5594 00000100:00100000:2.0:1789760733.863755:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718087663350 time 718130523561 state 0x2 ref 0 00000100:00100000:1.0:1789760733.867583:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760733.867719:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495792384 00000100:00100000:0.0:1789760733.867734:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495792384 00000100:00100000:0.0:1789760733.867810:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718087663350 time 718130523561 state 0x0 ref 1 00000100:00100000:0.0:1789760733.867829:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718130783977 time 718130523561 state 0x1 ref 1 00000100:00100000:0.0:1789760733.867869:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718130783977 time 718188585419 state 0x0 ref 1 00000100:00100000:0.0:1789760733.867884:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495792384:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760733.867950:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22798336, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb007 00010000:00010000:0.0:1789760733.867962:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545ffe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.867982:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760733.867988:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760733.867996:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545ffe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.868027:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545ffe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760733.919473:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972883 is committed 00002000:00010000:2.0:1789760733.919590:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22798336 -> 22802432 00010000:00010000:2.0:1789760733.919609:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545ffe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.919624:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545ffe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.919634:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545ffe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.919642:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760733.919648:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545ffe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.919658:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545ffe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.919693:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545ffe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.919702:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354545ffe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.919819:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495792384:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51930us (52243us total) trans 4294972883 rc 0/0 00000100:00100000:2.0:1789760733.919830:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5595 00000100:00100000:2.0:1789760733.919842:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718130783977 time 718188585419 state 0x2 ref 0 00000100:00100000:3.0:1789760733.929261:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760733.929426:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495792512 00000100:00100000:2.0:1789760733.929440:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495792512 00000100:00100000:2.0:1789760733.929518:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718130783977 time 718188585419 state 0x0 ref 1 00000100:00100000:2.0:1789760733.929539:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718188845835 time 718188585419 state 0x1 ref 1 00000100:00100000:2.0:1789760733.929577:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718188845835 time 718250293416 state 0x0 ref 1 00000100:00100000:2.0:1789760733.929593:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495792512:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760733.929665:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22802432, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb007 00010000:00010000:2.0:1789760733.929676:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546005 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.929698:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760733.929703:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.929711:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546005 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.929728:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546005 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760733.967351:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22802432 -> 22806528 00010000:00010000:2.0:1789760733.967382:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546005 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.967399:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546005 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.967410:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546005 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.967430:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760733.967437:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546005 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.967448:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546005 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.967475:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546005 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.967484:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546005 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.967612:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495792512:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38014us (38359us total) trans 4294972884 rc 0/0 00000100:00100000:2.0:1789760733.967626:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5596 00000100:00100000:2.0:1789760733.967641:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718188845835 time 718250293416 state 0x2 ref 0 00000001:00080000:0.0:1789760733.968704:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972884 is committed 00000100:00100000:0.0:1789760733.971885:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760733.971960:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495792640 00000100:00100000:2.0:1789760733.971975:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495792640 00000100:00100000:2.0:1789760733.972051:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718188845835 time 718250293416 state 0x0 ref 1 00000100:00100000:2.0:1789760733.972070:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718250553832 time 718250293416 state 0x1 ref 1 00000100:00100000:2.0:1789760733.972233:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718250553832 time 718292948140 state 0x0 ref 1 00000100:00100000:2.0:1789760733.972249:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03095180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495792640:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760733.972357:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22806528, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb007 00010000:00010000:2.0:1789760733.972368:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454600c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.972389:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760733.972394:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.972401:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454600c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.972417:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454600c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760734.010306:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972885 is committed 00002000:00010000:2.0:1789760734.010667:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22806528 -> 22810624 00010000:00010000:2.0:1789760734.010690:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454600c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.010707:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454600c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.010718:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454600c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.010727:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760734.010733:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454600c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.010757:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454600c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.010782:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454600c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.010791:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454600c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.010918:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495792640:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38665us (39042us total) trans 4294972885 rc 0/0 00000100:00100000:2.0:1789760734.010931:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5597 00000100:00100000:2.0:1789760734.010944:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718250553832 time 718292948140 state 0x2 ref 0 00000100:00100000:1.0:1789760734.016273:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760734.016435:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495792768 00000100:00100000:2.0:1789760734.016448:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495792768 00000100:00100000:2.0:1789760734.016532:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718250553832 time 718292948140 state 0x0 ref 1 00000100:00100000:2.0:1789760734.016552:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718293208556 time 718292948140 state 0x1 ref 1 00000100:00100000:2.0:1789760734.016581:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718293208556 time 718337298411 state 0x0 ref 1 00000100:00100000:2.0:1789760734.016592:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495792768:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760734.016646:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22810624, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb007 00010000:00010000:2.0:1789760734.016654:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546013 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.016671:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760734.016675:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.016680:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546013 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.016693:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546013 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.051818:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760734.067209:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972886 is committed 00002000:00010000:2.0:1789760734.067364:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22810624 -> 22814720 00010000:00010000:2.0:1789760734.067387:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546013 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.067404:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546013 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.067416:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546013 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.067426:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760734.067432:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546013 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.067443:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546013 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.067476:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546013 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.067485:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546013 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.067604:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495792768:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51005us (51341us total) trans 4294972886 rc 0/0 00000100:00100000:2.0:1789760734.067617:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5598 00000100:00100000:2.0:1789760734.067629:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718293208556 time 718337298411 state 0x2 ref 0 00000100:00100000:1.0:1789760734.071637:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760734.071755:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495792896 00000100:00100000:2.0:1789760734.071769:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495792896 00000100:00100000:2.0:1789760734.071913:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718293208556 time 718337298411 state 0x0 ref 1 00000100:00100000:2.0:1789760734.071933:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718337558827 time 718337298411 state 0x1 ref 1 00000100:00100000:2.0:1789760734.071967:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718337558827 time 718392683471 state 0x0 ref 1 00000100:00100000:2.0:1789760734.071982:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495792896:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760734.072051:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22814720, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb007 00010000:00010000:2.0:1789760734.072062:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454601a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.072155:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760734.072161:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.072169:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454601a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.072186:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454601a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760734.117472:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22814720 -> 22818816 00010000:00010000:3.0:1789760734.117503:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454601a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.117519:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454601a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.117530:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454601a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.117538:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760734.117545:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454601a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.117556:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454601a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.117583:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454601a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.117591:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454601a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.117721:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495792896:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45734us (46090us total) trans 4294972887 rc 0/0 00000100:00100000:3.0:1789760734.117734:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5599 00000100:00100000:3.0:1789760734.117746:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718337558827 time 718392683471 state 0x2 ref 0 00000001:00080000:1.0:1789760734.118586:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972887 is committed 00000100:00100000:1.0:1789760734.121501:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760734.121618:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495793024 00000100:00100000:2.0:1789760734.121630:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495793024 00000100:00100000:2.0:1789760734.121701:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718337558827 time 718392683471 state 0x0 ref 1 00000100:00100000:2.0:1789760734.121720:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718392943887 time 718392683471 state 0x1 ref 1 00000100:00100000:2.0:1789760734.121754:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718392943887 time 718442470292 state 0x0 ref 1 00000100:00100000:2.0:1789760734.121767:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495793024:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760734.121834:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22818816, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb007 00010000:00010000:2.0:1789760734.121845:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546021 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.121866:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760734.121870:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.121878:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546021 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.121901:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546021 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760734.162536:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972888 is committed 00002000:00010000:3.0:1789760734.163190:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22818816 -> 22822912 00010000:00010000:3.0:1789760734.163212:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546021 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.163229:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546021 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.163238:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546021 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.163251:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760734.163257:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546021 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.163267:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546021 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.163361:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546021 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.163371:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546021 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.163594:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495793024:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41822us (42156us total) trans 4294972888 rc 0/0 00000100:00100000:3.0:1789760734.163607:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5600 00000100:00100000:3.0:1789760734.163620:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718392943887 time 718442470292 state 0x2 ref 0 00000100:00100000:2.0:1789760734.170704:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760734.170856:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495793152 00000100:00100000:0.0:1789760734.170870:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495793152 00000100:00100000:0.0:1789760734.170962:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718392943887 time 718442470292 state 0x0 ref 1 00000100:00100000:0.0:1789760734.170982:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718442730708 time 718442470292 state 0x1 ref 1 00000100:00100000:0.0:1789760734.171020:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718442730708 time 718491736144 state 0x0 ref 1 00000100:00100000:0.0:1789760734.171369:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92901237ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495793152:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760734.171442:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22822912, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb007 00010000:00010000:0.0:1789760734.171539:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546028 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.171560:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760734.171565:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760734.171572:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546028 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.171591:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546028 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760734.223019:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972889 is committed 00002000:00010000:2.0:1789760734.223374:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22822912 -> 22827008 00010000:00010000:2.0:1789760734.223396:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546028 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.223413:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546028 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.223423:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546028 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.223431:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760734.223437:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546028 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.223448:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546028 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.223474:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546028 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.223481:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546028 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.223660:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92901237ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495793152:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52619us (52962us total) trans 4294972889 rc 0/0 00000100:00100000:2.0:1789760734.223673:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5601 00000100:00100000:2.0:1789760734.223686:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718442730708 time 718491736144 state 0x2 ref 0 00000100:00100000:1.0:1789760734.228395:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760734.228495:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495793280 00000100:00100000:3.0:1789760734.228509:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495793280 00000100:00100000:3.0:1789760734.228582:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718442730708 time 718491736144 state 0x0 ref 1 00000100:00100000:3.0:1789760734.228601:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718491996560 time 718491736144 state 0x1 ref 1 00000100:00100000:3.0:1789760734.228634:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718491996560 time 718549350214 state 0x0 ref 1 00000100:00100000:3.0:1789760734.228651:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495793280:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760734.228719:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22827008, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb007 00010000:00010000:3.0:1789760734.228730:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454602f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.228750:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760734.228756:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.228774:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454602f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.228792:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454602f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760734.280834:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972890 is committed 00002000:00010000:2.0:1789760734.280974:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22827008 -> 22831104 00010000:00010000:2.0:1789760734.281000:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454602f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.281017:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454602f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.281028:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454602f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.281037:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760734.281043:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454602f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.281054:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454602f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.281080:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454602f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.281108:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454602f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.281248:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495793280:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52591us (52860us total) trans 4294972890 rc 0/0 00000100:00100000:2.0:1789760734.281261:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5602 00000100:00100000:2.0:1789760734.281274:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718491996560 time 718549350214 state 0x2 ref 0 00000100:00100000:0.0:1789760734.294885:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760734.295071:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495793408 00000100:00100000:3.0:1789760734.295084:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495793408 00000100:00100000:3.0:1789760734.295180:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718491996560 time 718549350214 state 0x0 ref 1 00000100:00100000:3.0:1789760734.295200:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718549610630 time 718549350214 state 0x1 ref 1 00000100:00100000:3.0:1789760734.295239:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718549610630 time 718615954638 state 0x0 ref 1 00000100:00100000:3.0:1789760734.295252:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495793408:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760734.295370:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22831104, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb007 00010000:00010000:3.0:1789760734.295381:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546036 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.295403:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760734.295408:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.295415:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546036 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.295432:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546036 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760734.374169:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22831104 -> 22835200 00010000:00010000:3.0:1789760734.374197:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546036 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.374213:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546036 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.374225:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546036 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.374239:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760734.374244:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546036 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.374254:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546036 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.374301:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546036 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.374310:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546036 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.374439:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495793408:12345-192.168.203.9@tcp:4:dd.0 Request processed in 79182us (79562us total) trans 4294972891 rc 0/0 00000100:00100000:3.0:1789760734.374453:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5603 00000100:00100000:3.0:1789760734.374465:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718549610630 time 718615954638 state 0x2 ref 0 00000001:00080000:3.0:1789760734.374535:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972891 is committed 00000100:00100000:0.0:1789760734.381618:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760734.383154:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495793536 00000100:00100000:2.0:1789760734.383173:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495793536 00000100:00100000:1.0:1789760734.384007:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718549610630 time 718615954638 state 0x0 ref 1 00000100:00100000:1.0:1789760734.384029:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718616215054 time 718615954638 state 0x1 ref 1 00000100:00100000:1.0:1789760734.384076:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718616215054 time 718704791226 state 0x0 ref 1 00000100:00100000:1.0:1789760734.384091:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495793536:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760734.384170:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22835200, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb007 00010000:00010000:1.0:1789760734.384181:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454603d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.384206:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760734.384211:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.384220:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454603d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.384237:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454603d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760734.495954:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972892 is committed 00002000:00010000:1.0:1789760734.496147:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22835200 -> 22839296 00010000:00010000:1.0:1789760734.496169:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454603d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.496185:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454603d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.496200:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454603d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.496208:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760734.496214:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454603d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.496225:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454603d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.496251:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454603d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.496260:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454603d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760734.496555:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495793536:12345-192.168.203.9@tcp:4:dd.0 Request processed in 112459us (114942us total) trans 4294972892 rc 0/0 00000100:00100000:1.0:1789760734.496569:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5604 00000100:00100000:1.0:1789760734.496583:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718616215054 time 718704791226 state 0x2 ref 0 00000100:00100000:2.0:1789760734.501524:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760734.506186:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495793664 00000100:00100000:3.0:1789760734.506206:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495793664 00000100:00100000:3.0:1789760734.506318:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718616215054 time 718704791226 state 0x0 ref 1 00000100:00100000:3.0:1789760734.506339:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718705051642 time 718704791226 state 0x1 ref 1 00000100:00100000:3.0:1789760734.506376:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718705051642 time 718827092542 state 0x0 ref 1 00000100:00100000:3.0:1789760734.506391:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495793664:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760734.506460:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22839296, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb007 00010000:00010000:3.0:1789760734.506470:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546044 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.506491:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760734.506497:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.506504:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546044 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.506522:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546044 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760734.562814:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972893 is committed 00002000:00010000:2.0:1789760734.563006:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22839296 -> 22843392 00010000:00010000:2.0:1789760734.563028:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546044 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.563044:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546044 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.563055:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546044 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.563064:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760734.563080:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546044 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.563106:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546044 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.563132:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546044 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.563141:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546044 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.563267:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495793664:12345-192.168.203.9@tcp:4:dd.0 Request processed in 56873us (61752us total) trans 4294972893 rc 0/0 00000100:00100000:2.0:1789760734.563321:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5605 00000100:00100000:2.0:1789760734.563335:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718705051642 time 718827092542 state 0x2 ref 0 00000100:00100000:2.0:1789760734.578187:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760734.579104:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495793792 00000100:00100000:3.0:1789760734.579120:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495793792 00000100:00100000:3.0:1789760734.579201:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718705051642 time 718827092542 state 0x0 ref 1 00000100:00100000:3.0:1789760734.579220:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718827352958 time 718827092542 state 0x1 ref 1 00000100:00100000:3.0:1789760734.579259:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718827352958 time 718899975880 state 0x0 ref 1 00000100:00100000:3.0:1789760734.579274:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495793792:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760734.579391:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22843392, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb007 00010000:00010000:3.0:1789760734.579402:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454604b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.579424:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760734.579430:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.579438:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454604b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.579456:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454604b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760734.641731:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22843392 -> 22847488 00010000:00010000:3.0:1789760734.641757:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454604b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.641773:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454604b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.641785:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454604b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.641793:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760734.641799:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454604b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.641810:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454604b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.641835:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454604b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.641844:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454604b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.641973:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495793792:12345-192.168.203.9@tcp:4:dd.0 Request processed in 62694us (63793us total) trans 4294972894 rc 0/0 00000100:00100000:3.0:1789760734.641991:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5606 00000100:00100000:3.0:1789760734.642003:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718827352958 time 718899975880 state 0x2 ref 0 00000001:00080000:2.0:1789760734.643325:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972894 is committed 00000100:00100000:3.0:1789760734.645232:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760734.645526:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495793920 00000100:00100000:1.0:1789760734.645540:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495793920 00000100:00100000:1.0:1789760734.645622:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718827352958 time 718899975880 state 0x0 ref 1 00000100:00100000:1.0:1789760734.645644:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718900236296 time 718899975880 state 0x1 ref 1 00000100:00100000:1.0:1789760734.645758:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718900236296 time 718966473945 state 0x0 ref 1 00000100:00100000:1.0:1789760734.645774:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495793920:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760734.645851:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22847488, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb007 00010000:00010000:1.0:1789760734.645863:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546052 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.645884:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760734.645890:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.645899:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546052 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.645918:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546052 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760734.708264:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972895 is committed 00002000:00010000:2.0:1789760734.708505:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22847488 -> 22851584 00002000:00010000:2.0:1789760734.708511:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760733 -> 1789760734 00002000:00010000:2.0:1789760734.708514:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760733 -> 1789760734 00010000:00010000:2.0:1789760734.708531:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546052 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.708547:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546052 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.708563:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546052 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.708572:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760734.708579:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546052 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.708589:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546052 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.708619:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546052 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.708628:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546052 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.708756:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495793920:12345-192.168.203.9@tcp:4:dd.0 Request processed in 62978us (63531us total) trans 4294972895 rc 0/0 00000100:00100000:2.0:1789760734.708769:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5607 00000100:00100000:2.0:1789760734.708784:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718900236296 time 718966473945 state 0x2 ref 0 00000100:00100000:0.0:1789760734.715465:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760734.715633:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495794048 00000100:00100000:2.0:1789760734.715646:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495794048 00000100:00100000:2.0:1789760734.715719:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718900236296 time 718966473945 state 0x0 ref 1 00000100:00100000:2.0:1789760734.715738:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718966734361 time 718966473945 state 0x1 ref 1 00000100:00100000:2.0:1789760734.715775:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718966734361 time 719036490984 state 0x0 ref 1 00000100:00100000:2.0:1789760734.715789:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495794048:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760734.715855:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22851584, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb007 00010000:00010000:2.0:1789760734.715866:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546059 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.715887:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760734.715893:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.715901:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546059 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.715917:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546059 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760734.769365:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972896 is committed 00002000:00010000:2.0:1789760734.769659:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22851584 -> 22855680 00010000:00010000:2.0:1789760734.769683:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546059 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.769700:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546059 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.769711:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546059 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.769720:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760734.769728:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546059 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.769739:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546059 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.769766:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546059 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.769774:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546059 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.770355:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495794048:12345-192.168.203.9@tcp:4:dd.0 Request processed in 54556us (54891us total) trans 4294972896 rc 0/0 00000100:00100000:2.0:1789760734.770377:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5608 00000100:00100000:2.0:1789760734.770392:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718966734361 time 719036490984 state 0x2 ref 0 00000100:00100000:0.0:1789760734.776172:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760734.776329:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495794176 00000100:00100000:3.0:1789760734.776341:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495794176 00000100:00100000:3.0:1789760734.776414:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 718966734361 time 719036490984 state 0x0 ref 1 00000100:00100000:3.0:1789760734.776434:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719036751400 time 719036490984 state 0x1 ref 1 00000100:00100000:3.0:1789760734.776481:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719036751400 time 719097197180 state 0x0 ref 1 00000100:00100000:3.0:1789760734.776496:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495794176:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760734.776559:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22855680, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb007 00010000:00010000:3.0:1789760734.776572:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546060 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.776592:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760734.776598:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.776606:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546060 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.776622:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546060 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760734.821419:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972897 is committed 00002000:00010000:0.0:1789760734.821547:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22855680 -> 22859776 00010000:00010000:0.0:1789760734.821567:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546060 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.821583:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546060 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.821656:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546060 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.821665:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760734.821670:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546060 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.821680:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546060 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.821706:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546060 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.821719:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546060 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760734.821909:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495794176:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45409us (45745us total) trans 4294972897 rc 0/0 00000100:00100000:0.0:1789760734.821922:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5609 00000100:00100000:0.0:1789760734.821935:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719036751400 time 719097197180 state 0x2 ref 0 00000100:00100000:3.0:1789760734.825492:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760734.825782:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495794304 00000100:00100000:2.0:1789760734.825796:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495794304 00000100:00100000:2.0:1789760734.825874:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719036751400 time 719097197180 state 0x0 ref 1 00000100:00100000:2.0:1789760734.825892:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719097457596 time 719097197180 state 0x1 ref 1 00000100:00100000:2.0:1789760734.825927:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719097457596 time 719146643320 state 0x0 ref 1 00000100:00100000:2.0:1789760734.825941:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04931f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495794304:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760734.826003:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22859776, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb007 00010000:00010000:2.0:1789760734.826014:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546067 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.826033:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760734.826038:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.826046:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546067 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.826062:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546067 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760734.868951:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972898 is committed 00002000:00010000:2.0:1789760734.869106:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22859776 -> 22863872 00010000:00010000:2.0:1789760734.869126:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546067 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.869141:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546067 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.869149:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546067 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.869156:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760734.869162:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546067 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.869171:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546067 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.869199:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546067 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.869206:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546067 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.869364:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495794304:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43418us (43879us total) trans 4294972898 rc 0/0 00000100:00100000:2.0:1789760734.869376:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5610 00000100:00100000:2.0:1789760734.869389:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719097457596 time 719146643320 state 0x2 ref 0 00000100:00100000:2.0:1789760734.873038:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760734.873207:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495794432 00000100:00100000:3.0:1789760734.873220:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495794432 00000100:00100000:3.0:1789760734.873544:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719097457596 time 719146643320 state 0x0 ref 1 00000100:00100000:3.0:1789760734.873563:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719146903736 time 719146643320 state 0x1 ref 1 00000100:00100000:2.0:1789760734.873679:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719146903736 time 719194392125 state 0x0 ref 1 00000100:00100000:2.0:1789760734.873697:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903639f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495794432:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760734.873757:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22863872, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb007 00010000:00010000:2.0:1789760734.873768:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454606e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.873788:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760734.873794:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.873801:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454606e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.873818:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454606e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760734.909935:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972899 is committed 00002000:00010000:2.0:1789760734.910173:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22863872 -> 22867968 00010000:00010000:2.0:1789760734.910193:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454606e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.910209:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454606e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.910220:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454606e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.910229:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760734.910235:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454606e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.910246:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454606e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.910272:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454606e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.910306:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454606e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.910502:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903639f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495794432:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36801us (37469us total) trans 4294972899 rc 0/0 00000100:00100000:2.0:1789760734.910521:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5611 00000100:00100000:2.0:1789760734.910533:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719146903736 time 719194392125 state 0x2 ref 0 00000100:00100000:0.0:1789760734.914929:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760734.915191:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495794560 00000100:00100000:3.0:1789760734.915204:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495794560 00000100:00100000:3.0:1789760734.915577:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719146903736 time 719194392125 state 0x0 ref 1 00000100:00100000:3.0:1789760734.915606:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719194652541 time 719194392125 state 0x1 ref 1 00000100:00100000:2.0:1789760734.915691:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719194652541 time 719236405203 state 0x0 ref 1 00000100:00100000:2.0:1789760734.915709:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495794560:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760734.915770:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22867968, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb007 00010000:00010000:2.0:1789760734.915781:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546075 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.915800:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760734.915806:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.915814:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546075 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.915830:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546075 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760734.950222:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972900 is committed 00002000:00010000:2.0:1789760734.950414:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22867968 -> 22872064 00010000:00010000:2.0:1789760734.950435:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546075 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.950450:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546075 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.950523:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546075 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.950531:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760734.950593:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546075 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.950605:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546075 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.950628:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546075 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.950637:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546075 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.950771:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495794560:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35058us (35849us total) trans 4294972900 rc 0/0 00000100:00100000:2.0:1789760734.950782:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5612 00000100:00100000:2.0:1789760734.950795:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719194652541 time 719236405203 state 0x2 ref 0 00000100:00100000:0.0:1789760734.956267:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760734.957059:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495794688 00000100:00100000:2.0:1789760734.957144:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495794688 00000100:00100000:2.0:1789760734.957221:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719194652541 time 719236405203 state 0x0 ref 1 00000100:00100000:2.0:1789760734.957237:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719236665619 time 719236405203 state 0x1 ref 1 00000100:00100000:2.0:1789760734.957336:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719236665619 time 719278051150 state 0x0 ref 1 00000100:00100000:2.0:1789760734.957354:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495794688:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760734.957442:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22872064, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb007 00010000:00010000:2.0:1789760734.957453:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454607c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.957475:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760734.957480:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.957488:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454607c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.957504:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454607c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760734.998430:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972901 is committed 00002000:00010000:1.0:1789760734.998603:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22872064 -> 22876160 00010000:00010000:1.0:1789760734.998623:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454607c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.998638:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454607c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.998648:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454607c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.998663:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760734.998670:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454607c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.998680:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454607c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.998701:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454607c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.998709:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454607c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760735.000965:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495794688:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41463us (42563us total) trans 4294972901 rc 0/0 00000100:00100000:1.0:1789760735.000985:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5613 00000100:00100000:1.0:1789760735.001058:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719236665619 time 719278051150 state 0x2 ref 0 00000100:00100000:0.0:1789760735.002940:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760735.003214:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495794816 00000100:00100000:2.0:1789760735.003228:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495794816 00000100:00100000:2.0:1789760735.003539:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719236665619 time 719278051150 state 0x0 ref 1 00000100:00100000:2.0:1789760735.003558:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719278311566 time 719278051150 state 0x1 ref 1 00000100:00100000:2.0:1789760735.003683:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719278311566 time 719324398899 state 0x0 ref 1 00000100:00100000:2.0:1789760735.003700:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370bd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495794816:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760735.003771:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22876160, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb007 00010000:00010000:2.0:1789760735.003781:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546083 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.003805:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760735.003810:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.003910:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546083 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.003929:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546083 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760735.005983:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760735.053717:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972902 is committed 00002000:00010000:2.0:1789760735.053955:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22876160 -> 22880256 00010000:00010000:2.0:1789760735.053977:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546083 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.053993:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546083 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.054004:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546083 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.054013:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760735.054019:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546083 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.054028:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546083 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.054059:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546083 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.054068:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546083 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760735.054204:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370bd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495794816:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50501us (51269us total) trans 4294972902 rc 0/0 00000100:00100000:2.0:1789760735.054217:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5614 00000100:00100000:2.0:1789760735.054230:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719278311566 time 719324398899 state 0x2 ref 0 00000100:00100000:2.0:1789760735.057517:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760735.057697:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495794944 00000100:00100000:0.0:1789760735.057710:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495794944 00000100:00100000:0.0:1789760735.057787:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719278311566 time 719324398899 state 0x0 ref 1 00000100:00100000:0.0:1789760735.057806:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719324659315 time 719324398899 state 0x1 ref 1 00000100:00100000:0.0:1789760735.057908:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719324659315 time 719378623512 state 0x0 ref 1 00000100:00100000:0.0:1789760735.057924:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495794944:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760735.057992:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22880256, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb007 00010000:00010000:0.0:1789760735.058007:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454608a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.058027:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760735.058032:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760735.058040:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454608a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.058056:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454608a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.073640:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760735.106031:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972903 is committed 00002000:00010000:3.0:1789760735.106189:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22880256 -> 22884352 00010000:00010000:3.0:1789760735.106208:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454608a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.106224:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454608a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.106235:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454608a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.106244:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760735.106251:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454608a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.106262:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454608a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.106319:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454608a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.106329:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454608a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.106462:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495794944:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48534us (48953us total) trans 4294972903 rc 0/0 00000100:00100000:3.0:1789760735.106474:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5615 00000100:00100000:3.0:1789760735.106486:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719324659315 time 719378623512 state 0x2 ref 0 00000100:00100000:0.0:1789760735.110353:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760735.110554:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495795072 00000100:00100000:1.0:1789760735.110567:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495795072 00000100:00100000:1.0:1789760735.110649:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719324659315 time 719378623512 state 0x0 ref 1 00000100:00100000:1.0:1789760735.110667:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719378883928 time 719378623512 state 0x1 ref 1 00000100:00100000:1.0:1789760735.110707:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719378883928 time 719431423426 state 0x0 ref 1 00000100:00100000:1.0:1789760735.110720:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495795072:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760735.110782:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22884352, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb007 00010000:00010000:1.0:1789760735.110792:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546091 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.110811:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760735.110989:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760735.110998:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546091 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.111015:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546091 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.158266:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972904 is committed 00002000:00010000:1.0:1789760735.158390:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22884352 -> 22888448 00010000:00010000:1.0:1789760735.158478:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546091 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.158496:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546091 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.158506:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546091 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.158521:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760735.158528:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546091 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.158538:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546091 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.158562:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546091 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.158693:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546091 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760735.158811:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495795072:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48085us (48466us total) trans 4294972904 rc 0/0 00000100:00100000:1.0:1789760735.158940:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5616 00000100:00100000:1.0:1789760735.158953:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719378883928 time 719431423426 state 0x2 ref 0 00000100:00100000:1.0:1789760735.163304:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760735.163471:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495795200 00000100:00100000:2.0:1789760735.163484:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495795200 00000100:00100000:2.0:1789760735.163557:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719378883928 time 719431423426 state 0x0 ref 1 00000100:00100000:2.0:1789760735.163697:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719431683842 time 719431423426 state 0x1 ref 1 00000100:00100000:2.0:1789760735.163770:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719431683842 time 719484485741 state 0x0 ref 1 00000100:00100000:2.0:1789760735.163786:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495795200:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760735.163915:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22888448, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb007 00010000:00010000:2.0:1789760735.163926:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546098 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.163947:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760735.163953:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.163960:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546098 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.163977:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546098 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.225223:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972905 is committed 00002000:00010000:0.0:1789760735.225571:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22888448 -> 22892544 00010000:00010000:0.0:1789760735.225592:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546098 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.225609:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546098 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.225620:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546098 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.225629:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760735.225634:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546098 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.225644:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546098 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.225670:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546098 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.225679:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546098 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760735.225804:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495795200:12345-192.168.203.9@tcp:4:dd.0 Request processed in 62014us (62532us total) trans 4294972905 rc 0/0 00000100:00100000:0.0:1789760735.225815:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5617 00000100:00100000:0.0:1789760735.225828:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719431683842 time 719484485741 state 0x2 ref 0 00000100:00100000:2.0:1789760735.228798:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760735.228917:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495795328 00000100:00100000:3.0:1789760735.228933:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495795328 00000100:00100000:3.0:1789760735.229019:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719431683842 time 719484485741 state 0x0 ref 1 00000100:00100000:3.0:1789760735.229038:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719484746157 time 719484485741 state 0x1 ref 1 00000100:00100000:3.0:1789760735.229079:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719484746157 time 719549786331 state 0x0 ref 1 00000100:00100000:3.0:1789760735.229101:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495795328:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760735.229162:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22892544, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb007 00010000:00010000:3.0:1789760735.229173:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454609f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.229192:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760735.229197:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.229205:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454609f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.229220:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454609f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760735.278960:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972906 is committed 00002000:00010000:3.0:1789760735.279218:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22892544 -> 22896640 00010000:00010000:3.0:1789760735.279240:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454609f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.279256:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454609f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.279267:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454609f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.279276:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760735.279352:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454609f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.279364:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454609f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.279390:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454609f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.279398:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454609f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.279686:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495795328:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50587us (50895us total) trans 4294972906 rc 0/0 00000100:00100000:3.0:1789760735.279701:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5618 00000100:00100000:3.0:1789760735.279715:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719484746157 time 719549786331 state 0x2 ref 0 00000100:00100000:2.0:1789760735.283685:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760735.283808:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495795456 00000100:00100000:1.0:1789760735.283823:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495795456 00000100:00100000:1.0:1789760735.283899:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719484746157 time 719549786331 state 0x0 ref 1 00000100:00100000:1.0:1789760735.283917:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719550046747 time 719549786331 state 0x1 ref 1 00000100:00100000:1.0:1789760735.283987:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719550046747 time 719604703006 state 0x0 ref 1 00000100:00100000:1.0:1789760735.284001:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495795456:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760735.284067:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22896640, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb007 00010000:00010000:1.0:1789760735.284096:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460a6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.284116:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760735.284121:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760735.284195:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.284214:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760735.357257:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972907 is committed 00002000:00010000:1.0:1789760735.357502:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22896640 -> 22900736 00010000:00010000:1.0:1789760735.357523:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.357539:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460a6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.357548:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.357556:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760735.357563:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.357574:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460a6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.357598:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460a6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.357607:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460a6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760735.357741:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495795456:12345-192.168.203.9@tcp:4:dd.0 Request processed in 73734us (74065us total) trans 4294972907 rc 0/0 00000100:00100000:1.0:1789760735.357754:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5619 00000100:00100000:1.0:1789760735.357767:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719550046747 time 719604703006 state 0x2 ref 0 00000100:00100000:1.0:1789760735.361690:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760735.361962:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495795584 00000100:00100000:0.0:1789760735.361979:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495795584 00000100:00100000:0.0:1789760735.362150:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719550046747 time 719604703006 state 0x0 ref 1 00000100:00100000:0.0:1789760735.362175:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719604963422 time 719604703006 state 0x1 ref 1 00000100:00100000:0.0:1789760735.362217:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719604963422 time 719682933268 state 0x0 ref 1 00000100:00100000:0.0:1789760735.362232:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495795584:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760735.362530:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22900736, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb007 00010000:00010000:0.0:1789760735.362542:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545460ad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.362563:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760735.362697:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760735.362706:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545460ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.362724:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545460ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760735.417714:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972908 is committed 00002000:00010000:1.0:1789760735.417808:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22900736 -> 22904832 00010000:00010000:1.0:1789760735.417832:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545460ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.417849:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545460ad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.417859:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545460ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.417872:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760735.417878:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545460ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.417888:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545460ad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.417911:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545460ad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.417918:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545460ad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760735.418031:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495795584:12345-192.168.203.9@tcp:4:dd.0 Request processed in 55797us (56351us total) trans 4294972908 rc 0/0 00000100:00100000:1.0:1789760735.418042:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5620 00000100:00100000:1.0:1789760735.418056:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719604963422 time 719682933268 state 0x2 ref 0 00000100:00100000:2.0:1789760735.421227:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760735.421363:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495795712 00000100:00100000:3.0:1789760735.421378:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495795712 00000100:00100000:3.0:1789760735.421463:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719604963422 time 719682933268 state 0x0 ref 1 00000100:00100000:3.0:1789760735.421482:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719683193684 time 719682933268 state 0x1 ref 1 00000100:00100000:3.0:1789760735.421520:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719683193684 time 719742235883 state 0x0 ref 1 00000100:00100000:3.0:1789760735.421535:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495795712:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760735.421600:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22904832, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb007 00010000:00010000:3.0:1789760735.421611:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545460b4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.421632:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760735.421638:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.421646:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545460b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.421662:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545460b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760735.463840:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22904832 -> 22908928 00010000:00010000:0.0:1789760735.463869:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545460b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.463885:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545460b4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.463895:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545460b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.463903:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760735.463908:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545460b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.463917:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545460b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.463943:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545460b4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.463955:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545460b4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760735.464148:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495795712:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42608us (42928us total) trans 4294972909 rc 0/0 00000100:00100000:0.0:1789760735.464161:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5621 00000100:00100000:0.0:1789760735.464173:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719683193684 time 719742235883 state 0x2 ref 0 00000001:00080000:1.0:1789760735.465135:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972909 is committed 00000100:00100000:2.0:1789760735.470475:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760735.470603:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495795840 00000100:00100000:1.0:1789760735.470618:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495795840 00000100:00100000:1.0:1789760735.470702:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719683193684 time 719742235883 state 0x0 ref 1 00000100:00100000:1.0:1789760735.470722:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719742496299 time 719742235883 state 0x1 ref 1 00000100:00100000:1.0:1789760735.470764:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719742496299 time 719791481201 state 0x0 ref 1 00000100:00100000:1.0:1789760735.470780:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495795840:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760735.470850:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22908928, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb007 00010000:00010000:1.0:1789760735.470861:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460bb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.470883:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760735.470888:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760735.470897:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.470913:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.520373:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f01835c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520305920:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760735.520492:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760735.520752:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520305920 00000100:00100000:1.0:1789760735.520763:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520305920 00000100:00100000:1.0:1789760735.520844:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 197 00000100:00100000:1.0:1789760735.520855:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04933480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520305920:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760735.520885:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528547840 pending 1703936 free 3835691008 avail 3818913792 00000100:00100000:1.0:1789760735.520946:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04933480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520305920:12345-0@lo:13:osp-pre-0-0.0 Request processed in 92us (453us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760735.520953:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 197 00000100:00100000:1.0:1789760735.521014:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f01835c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520305920:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1789760735.521530:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789760735.579654:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22908928 -> 22913024 00010000:00010000:0.0:1789760735.579685:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.579702:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460bb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.579712:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.579721:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760735.579728:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.579741:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460bb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.579772:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460bb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.579782:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460bb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760735.579910:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495795840:12345-192.168.203.9@tcp:4:dd.0 Request processed in 109123us (109494us total) trans 4294972910 rc 0/0 00000100:00100000:0.0:1789760735.579923:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5622 00000100:00100000:0.0:1789760735.579937:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719742496299 time 719791481201 state 0x2 ref 0 00000001:00080000:1.0:1789760735.580357:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972910 is committed 00000100:00100000:2.0:1789760735.586322:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760735.586462:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495795968 00000100:00100000:1.0:1789760735.586476:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495795968 00000100:00100000:1.0:1789760735.586556:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719742496299 time 719791481201 state 0x0 ref 1 00000100:00100000:1.0:1789760735.586578:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719791741617 time 719791481201 state 0x1 ref 1 00000100:00100000:1.0:1789760735.586657:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719791741617 time 719907372706 state 0x0 ref 1 00000100:00100000:1.0:1789760735.586673:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495795968:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760735.586744:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22913024, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb007 00010000:00010000:1.0:1789760735.586756:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545460c2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.586778:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760735.586784:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760735.586793:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545460c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.586810:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545460c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760735.636748:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22913024 -> 22917120 00010000:00010000:1.0:1789760735.636776:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545460c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.636792:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545460c2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.636803:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545460c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.636812:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760735.636819:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545460c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.636831:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545460c2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.636859:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545460c2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.636869:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545460c2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760735.636999:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495795968:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50321us (50731us total) trans 4294972911 rc 0/0 00000100:00100000:1.0:1789760735.637012:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5623 00000100:00100000:1.0:1789760735.637025:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719791741617 time 719907372706 state 0x2 ref 0 00000001:00080000:2.0:1789760735.637242:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972911 is committed 00000100:00100000:0.0:1789760735.642346:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760735.642538:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495796096 00000100:00100000:3.0:1789760735.642552:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495796096 00000100:00100000:3.0:1789760735.642629:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719791741617 time 719907372706 state 0x0 ref 1 00000100:00100000:3.0:1789760735.642649:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719907633122 time 719907372706 state 0x1 ref 1 00000100:00100000:3.0:1789760735.642722:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719907633122 time 719963438335 state 0x0 ref 1 00000100:00100000:3.0:1789760735.642737:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495796096:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760735.642802:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22917120, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb007 00010000:00010000:3.0:1789760735.642813:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545460c9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.642834:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760735.642840:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.642849:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545460c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.642868:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545460c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760735.696163:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22917120 -> 22921216 00002000:00010000:1.0:1789760735.696174:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760734 -> 1789760735 00002000:00010000:1.0:1789760735.696177:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760734 -> 1789760735 00010000:00010000:1.0:1789760735.696194:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545460c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.696208:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545460c9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.696218:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545460c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.696226:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760735.696232:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545460c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.696242:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545460c9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.696272:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545460c9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.696313:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545460c9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760735.696444:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495796096:12345-192.168.203.9@tcp:4:dd.0 Request processed in 53702us (54176us total) trans 4294972912 rc 0/0 00000100:00100000:1.0:1789760735.696456:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5624 00000100:00100000:1.0:1789760735.696468:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719907633122 time 719963438335 state 0x2 ref 0 00000001:00080000:2.0:1789760735.696872:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972912 is committed 00000100:00100000:0.0:1789760735.701598:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760735.701716:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495796224 00000100:00100000:1.0:1789760735.701730:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495796224 00000100:00100000:1.0:1789760735.701805:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719907633122 time 719963438335 state 0x0 ref 1 00000100:00100000:1.0:1789760735.701824:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719963698751 time 719963438335 state 0x1 ref 1 00000100:00100000:1.0:1789760735.701913:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719963698751 time 720022628513 state 0x0 ref 1 00000100:00100000:1.0:1789760735.701929:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495796224:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760735.701997:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22921216, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb007 00010000:00010000:1.0:1789760735.702010:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545460d0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.702029:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760735.702034:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760735.702043:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545460d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.702060:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545460d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760735.764428:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22921216 -> 22925312 00010000:00010000:2.0:1789760735.764457:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545460d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.764475:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545460d0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.764485:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545460d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.764493:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760735.764499:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545460d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.764517:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545460d0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.764544:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545460d0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.764553:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545460d0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760735.764699:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495796224:12345-192.168.203.9@tcp:4:dd.0 Request processed in 62765us (63169us total) trans 4294972913 rc 0/0 00000100:00100000:2.0:1789760735.764713:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5625 00000100:00100000:2.0:1789760735.764727:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719963698751 time 720022628513 state 0x2 ref 0 00000001:00080000:2.0:1789760735.764824:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972913 is committed 00000100:00100000:0.0:1789760735.769273:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760735.769569:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495796352 00000100:00100000:1.0:1789760735.769581:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495796352 00000100:00100000:1.0:1789760735.769656:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 719963698751 time 720022628513 state 0x0 ref 1 00000100:00100000:1.0:1789760735.769677:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720022888929 time 720022628513 state 0x1 ref 1 00000100:00100000:1.0:1789760735.769714:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720022888929 time 720090430182 state 0x0 ref 1 00000100:00100000:1.0:1789760735.769729:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495796352:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760735.769802:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22925312, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb007 00010000:00010000:1.0:1789760735.769814:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460d7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.769836:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760735.769843:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760735.769851:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.769868:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760735.846039:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972914 is committed 00002000:00010000:0.0:1789760735.846878:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22925312 -> 22929408 00010000:00010000:0.0:1789760735.846901:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.846914:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460d7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.846925:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.846932:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760735.846939:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.846950:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460d7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.846976:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460d7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.846985:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545460d7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760735.847595:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495796352:12345-192.168.203.9@tcp:4:dd.0 Request processed in 77859us (78330us total) trans 4294972914 rc 0/0 00000100:00100000:0.0:1789760735.847609:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5626 00000100:00100000:0.0:1789760735.847622:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720022888929 time 720090430182 state 0x2 ref 0 00000100:00100000:1.0:1789760735.850247:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760735.850346:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495796480 00000100:00100000:3.0:1789760735.850361:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495796480 00000100:00100000:3.0:1789760735.850432:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720022888929 time 720090430182 state 0x0 ref 1 00000100:00100000:3.0:1789760735.850451:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720090690598 time 720090430182 state 0x1 ref 1 00000100:00100000:3.0:1789760735.850490:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720090690598 time 720171206139 state 0x0 ref 1 00000100:00100000:3.0:1789760735.850505:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495796480:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760735.850566:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22929408, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb007 00010000:00010000:3.0:1789760735.850577:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545460de lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.850597:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760735.850602:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.850609:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545460de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.850625:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545460de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760735.924856:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972915 is committed 00002000:00010000:3.0:1789760735.924982:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22929408 -> 22933504 00010000:00010000:3.0:1789760735.925003:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545460de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.925017:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545460de lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.925031:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545460de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.925040:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760735.925046:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545460de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.925057:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545460de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.925103:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545460de lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.925112:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545460de lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.925235:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495796480:12345-192.168.203.9@tcp:4:dd.0 Request processed in 74726us (74996us total) trans 4294972915 rc 0/0 00000100:00100000:3.0:1789760735.925248:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5627 00000100:00100000:3.0:1789760735.925262:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720090690598 time 720171206139 state 0x2 ref 0 00000100:00100000:1.0:1789760735.933791:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760735.933860:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495796608 00000100:00100000:2.0:1789760735.933875:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495796608 00000100:00100000:2.0:1789760735.933955:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720090690598 time 720171206139 state 0x0 ref 1 00000100:00100000:2.0:1789760735.933977:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720171466555 time 720171206139 state 0x1 ref 1 00000100:00100000:2.0:1789760735.934015:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720171466555 time 720254731382 state 0x0 ref 1 00000100:00100000:2.0:1789760735.934029:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495796608:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760735.934116:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22933504, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb007 00010000:00010000:2.0:1789760735.934128:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545460e5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.934147:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760735.934152:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.934159:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545460e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.934175:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545460e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760735.977051:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22933504 -> 22937600 00010000:00010000:1.0:1789760735.977095:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545460e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.977111:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545460e5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.977121:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545460e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.977128:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760735.977134:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545460e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.977149:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545460e5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.977173:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545460e5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.977181:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545460e5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760735.977342:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495796608:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43307us (43556us total) trans 4294972916 rc 0/0 00000100:00100000:1.0:1789760735.977355:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5628 00000100:00100000:1.0:1789760735.977368:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720171466555 time 720254731382 state 0x2 ref 0 00000001:00080000:3.0:1789760735.977553:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972916 is committed 00000100:00100000:0.0:1789760735.982837:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760735.982951:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495796736 00000100:00100000:2.0:1789760735.982964:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495796736 00000100:00100000:2.0:1789760735.983040:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720171466555 time 720254731382 state 0x0 ref 1 00000100:00100000:2.0:1789760735.983059:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720254991798 time 720254731382 state 0x1 ref 1 00000100:00100000:2.0:1789760735.983210:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720254991798 time 720303925516 state 0x0 ref 1 00000100:00100000:2.0:1789760735.983226:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495796736:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760735.983331:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22937600, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb007 00010000:00010000:2.0:1789760735.983342:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545460ec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.983362:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760735.983369:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.983376:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545460ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.983393:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545460ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760736.042601:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972917 is committed 00002000:00010000:1.0:1789760736.042827:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22937600 -> 22941696 00010000:00010000:1.0:1789760736.042848:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545460ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.042864:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545460ec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.042874:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545460ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.042882:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760736.042889:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545460ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.042899:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545460ec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.042924:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545460ec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.042932:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545460ec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.043164:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495796736:12345-192.168.203.9@tcp:4:dd.0 Request processed in 59933us (60334us total) trans 4294972917 rc 0/0 00000100:00100000:1.0:1789760736.043177:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5629 00000100:00100000:1.0:1789760736.043191:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720254991798 time 720303925516 state 0x2 ref 0 00000100:00100000:2.0:1789760736.050466:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760736.052491:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495796864 00000100:00100000:3.0:1789760736.052512:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495796864 00000100:00100000:3.0:1789760736.052605:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720254991798 time 720303925516 state 0x0 ref 1 00000100:00100000:3.0:1789760736.052629:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720304185932 time 720303925516 state 0x1 ref 1 00000100:00100000:3.0:1789760736.052729:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720304185932 time 720373445059 state 0x0 ref 1 00000100:00100000:3.0:1789760736.052745:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495796864:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760736.052813:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22941696, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb007 00010000:00010000:3.0:1789760736.052824:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545460f3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.052843:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760736.052849:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.052856:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545460f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.052871:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545460f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.098832:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760736.119535:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972918 is committed 00002000:00010000:1.0:1789760736.119742:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22941696 -> 22945792 00010000:00010000:1.0:1789760736.119762:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545460f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.119778:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545460f3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.119790:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545460f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.119804:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760736.119810:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545460f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.119820:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545460f3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.119847:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545460f3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.119855:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545460f3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.119978:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495796864:12345-192.168.203.9@tcp:4:dd.0 Request processed in 67229us (69520us total) trans 4294972918 rc 0/0 00000100:00100000:1.0:1789760736.119990:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5630 00000100:00100000:1.0:1789760736.120004:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720304185932 time 720373445059 state 0x2 ref 0 00000100:00100000:1.0:1789760736.129630:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760736.129841:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495796992 00000100:00100000:0.0:1789760736.129854:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495796992 00000100:00100000:0.0:1789760736.129930:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720304185932 time 720373445059 state 0x0 ref 1 00000100:00100000:0.0:1789760736.129950:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720373705475 time 720373445059 state 0x1 ref 1 00000100:00100000:0.0:1789760736.129988:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720373705475 time 720450704713 state 0x0 ref 1 00000100:00100000:0.0:1789760736.130003:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495796992:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760736.130070:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22945792, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb007 00010000:00010000:0.0:1789760736.130094:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545460fa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.130114:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760736.130120:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760736.130128:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545460fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.130150:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545460fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760736.184089:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972919 is committed 00002000:00010000:1.0:1789760736.184277:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22945792 -> 22949888 00010000:00010000:1.0:1789760736.184333:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545460fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.184350:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545460fa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.184368:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545460fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.184377:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760736.184383:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545460fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.184395:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545460fa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.184430:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545460fa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.184439:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545460fa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.184822:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495796992:12345-192.168.203.9@tcp:4:dd.0 Request processed in 54813us (55200us total) trans 4294972919 rc 0/0 00000100:00100000:1.0:1789760736.184835:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5631 00000100:00100000:1.0:1789760736.184848:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720373705475 time 720450704713 state 0x2 ref 0 00000100:00100000:0.0:1789760736.188538:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760736.188645:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495797120 00000100:00100000:1.0:1789760736.188659:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495797120 00000100:00100000:1.0:1789760736.188737:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720373705475 time 720450704713 state 0x0 ref 1 00000100:00100000:1.0:1789760736.188757:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720450965129 time 720450704713 state 0x1 ref 1 00000100:00100000:1.0:1789760736.188830:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720450965129 time 720509545168 state 0x0 ref 1 00000100:00100000:1.0:1789760736.188845:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495797120:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760736.188914:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22949888, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb007 00010000:00010000:1.0:1789760736.188925:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546101 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.188946:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760736.188951:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.188960:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546101 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.188978:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546101 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760736.227774:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972920 is committed 00002000:00010000:2.0:1789760736.227883:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22949888 -> 22953984 00010000:00010000:2.0:1789760736.227902:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546101 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.227916:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546101 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.227925:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546101 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.227932:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760736.227938:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546101 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.227948:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546101 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.227973:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546101 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.227984:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546101 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.228333:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495797120:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39432us (39873us total) trans 4294972920 rc 0/0 00000100:00100000:2.0:1789760736.228348:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5632 00000100:00100000:2.0:1789760736.228363:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720450965129 time 720509545168 state 0x2 ref 0 00000100:00100000:2.0:1789760736.232201:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760736.232755:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495797248 00000100:00100000:1.0:1789760736.232769:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495797248 00000100:00100000:1.0:1789760736.232854:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720450965129 time 720509545168 state 0x0 ref 1 00000100:00100000:1.0:1789760736.232875:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720509805584 time 720509545168 state 0x1 ref 1 00000100:00100000:1.0:1789760736.232960:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720509805584 time 720553675600 state 0x0 ref 1 00000100:00100000:1.0:1789760736.232974:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495797248:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760736.233046:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22953984, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb007 00010000:00010000:1.0:1789760736.233068:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546108 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.233100:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760736.233105:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.233113:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546108 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.233129:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546108 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760736.267884:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972921 is committed 00002000:00010000:1.0:1789760736.267949:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22953984 -> 22958080 00010000:00010000:1.0:1789760736.267970:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546108 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.267987:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546108 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.267996:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546108 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.268005:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760736.268011:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546108 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.268023:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546108 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.268056:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546108 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.268075:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546108 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.268212:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495797248:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35234us (36019us total) trans 4294972921 rc 0/0 00000100:00100000:1.0:1789760736.268224:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5633 00000100:00100000:1.0:1789760736.268237:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720509805584 time 720553675600 state 0x2 ref 0 00000100:00100000:0.0:1789760736.273222:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760736.273573:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495797376 00000100:00100000:1.0:1789760736.273596:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495797376 00000100:00100000:1.0:1789760736.273676:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720509805584 time 720553675600 state 0x0 ref 1 00000100:00100000:1.0:1789760736.273695:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720553936016 time 720553675600 state 0x1 ref 1 00000100:00100000:1.0:1789760736.273723:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720553936016 time 720594438818 state 0x0 ref 1 00000100:00100000:1.0:1789760736.273737:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495797376:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760736.273810:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22958080, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb007 00010000:00010000:1.0:1789760736.273822:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454610f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.273843:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760736.273849:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.273857:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454610f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.273873:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454610f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760736.325801:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972922 is committed 00002000:00010000:1.0:1789760736.325935:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22958080 -> 22962176 00010000:00010000:1.0:1789760736.325957:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454610f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.325972:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454610f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.325982:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454610f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.325990:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760736.325996:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454610f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.326007:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454610f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.326158:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454610f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.326168:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454610f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.326431:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495797376:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52688us (53217us total) trans 4294972922 rc 0/0 00000100:00100000:1.0:1789760736.326444:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5634 00000100:00100000:1.0:1789760736.326458:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720553936016 time 720594438818 state 0x2 ref 0 00000100:00100000:0.0:1789760736.330216:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760736.331379:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495797504 00000100:00100000:2.0:1789760736.331396:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495797504 00000100:00100000:2.0:1789760736.331921:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720553936016 time 720594438818 state 0x0 ref 1 00000100:00100000:2.0:1789760736.331943:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720594699234 time 720594438818 state 0x1 ref 1 00000100:00100000:2.0:1789760736.331985:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720594699234 time 720652701502 state 0x0 ref 1 00000100:00100000:2.0:1789760736.332000:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495797504:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760736.332090:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22962176, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb007 00010000:00010000:2.0:1789760736.332102:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546116 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.332122:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760736.332128:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.332135:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546116 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.332152:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546116 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760736.374160:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972923 is committed 00002000:00010000:2.0:1789760736.374417:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22962176 -> 22966272 00010000:00010000:2.0:1789760736.374440:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546116 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.374526:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546116 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.374538:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546116 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.374548:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760736.374555:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546116 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.374565:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546116 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.374590:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546116 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.374598:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546116 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.374720:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495797504:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42716us (44511us total) trans 4294972923 rc 0/0 00000100:00100000:2.0:1789760736.374733:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5635 00000100:00100000:2.0:1789760736.374745:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720594699234 time 720652701502 state 0x2 ref 0 00000100:00100000:2.0:1789760736.378008:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760736.378191:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495797632 00000100:00100000:1.0:1789760736.378203:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495797632 00000100:00100000:1.0:1789760736.378274:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720594699234 time 720652701502 state 0x0 ref 1 00000100:00100000:1.0:1789760736.378379:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720652961918 time 720652701502 state 0x1 ref 1 00000100:00100000:1.0:1789760736.378551:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720652961918 time 720699265197 state 0x0 ref 1 00000100:00100000:1.0:1789760736.378567:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495797632:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760736.378632:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22966272, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb007 00010000:00010000:1.0:1789760736.378644:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454611d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.378665:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760736.378670:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.378679:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454611d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.378695:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454611d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760736.416911:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22966272 -> 22970368 00010000:00010000:1.0:1789760736.416941:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454611d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.416957:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454611d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.416968:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454611d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.416976:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760736.416982:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454611d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.416993:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454611d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.417018:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454611d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.417026:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454611d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760736.417220:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972924 is committed 00000100:00100000:1.0:1789760736.417229:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+6:2357:x1876699495797632:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38658us (39230us total) trans 4294972924 rc 0/0 00000100:00100000:1.0:1789760736.417245:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5636 00000100:00100000:1.0:1789760736.417258:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720652961918 time 720699265197 state 0x2 ref 0 00000100:00100000:0.0:1789760736.422761:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760736.422912:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495797760 00000100:00100000:1.0:1789760736.422926:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495797760 00000100:00100000:1.0:1789760736.423009:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720652961918 time 720699265197 state 0x0 ref 1 00000100:00100000:1.0:1789760736.423028:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720699525613 time 720699265197 state 0x1 ref 1 00000100:00100000:1.0:1789760736.423264:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720699525613 time 720743978550 state 0x0 ref 1 00000100:00100000:1.0:1789760736.423320:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495797760:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760736.423389:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22970368, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb007 00010000:00010000:1.0:1789760736.423409:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546124 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.423429:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760736.423434:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.423442:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546124 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.423459:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546124 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760736.461094:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972925 is committed 00002000:00010000:2.0:1789760736.461244:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22970368 -> 22974464 00010000:00010000:2.0:1789760736.461265:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546124 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.461491:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546124 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.461504:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546124 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.461512:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760736.461518:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546124 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.461530:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546124 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.461556:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546124 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.461565:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546124 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.463247:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495797760:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39953us (40482us total) trans 4294972925 rc 0/0 00000100:00100000:3.0:1789760736.463264:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5637 00000100:00100000:3.0:1789760736.463319:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720699525613 time 720743978550 state 0x2 ref 0 00000100:00100000:3.0:1789760736.466853:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760736.466959:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495797888 00000100:00100000:2.0:1789760736.466973:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495797888 00000100:00100000:2.0:1789760736.467036:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720699525613 time 720743978550 state 0x0 ref 1 00000100:00100000:2.0:1789760736.467052:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720744238966 time 720743978550 state 0x1 ref 1 00000100:00100000:2.0:1789760736.467098:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720744238966 time 720787814624 state 0x0 ref 1 00000100:00100000:2.0:1789760736.467113:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04932680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495797888:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760736.467162:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22974464, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb007 00010000:00010000:2.0:1789760736.467171:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454612b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.467187:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760736.467192:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.467198:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454612b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.467212:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454612b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760736.505942:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972926 is committed 00002000:00010000:2.0:1789760736.506102:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22974464 -> 22978560 00010000:00010000:2.0:1789760736.506118:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454612b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.506130:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454612b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.506140:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454612b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.506147:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760736.506153:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454612b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.506162:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454612b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.506180:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454612b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.506187:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454612b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.506271:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04932680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495797888:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39158us (39429us total) trans 4294972926 rc 0/0 00000100:00100000:2.0:1789760736.506316:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5638 00000100:00100000:2.0:1789760736.506325:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720744238966 time 720787814624 state 0x2 ref 0 00000100:00100000:2.0:1789760736.510960:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760736.511043:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495798016 00000100:00100000:3.0:1789760736.511054:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495798016 00000100:00100000:3.0:1789760736.511128:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720744238966 time 720787814624 state 0x0 ref 1 00000100:00100000:3.0:1789760736.511143:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720788075040 time 720787814624 state 0x1 ref 1 00000100:00100000:3.0:1789760736.511175:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720788075040 time 720831891810 state 0x0 ref 1 00000100:00100000:3.0:1789760736.511187:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495798016:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760736.511230:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22978560, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb007 00010000:00010000:3.0:1789760736.511239:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546132 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.511255:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760736.511259:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.511265:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546132 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.511303:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546132 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760736.544159:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22978560 -> 22982656 00010000:00010000:3.0:1789760736.544178:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546132 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.544192:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546132 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.544202:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546132 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.544209:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760736.544213:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546132 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.544221:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546132 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.544239:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546132 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.544246:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546132 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.544374:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495798016:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33184us (33419us total) trans 4294972927 rc 0/0 00000100:00100000:3.0:1789760736.544384:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5639 00000100:00100000:3.0:1789760736.544394:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720788075040 time 720831891810 state 0x2 ref 0 00000001:00080000:3.0:1789760736.544435:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972927 is committed 00000100:00100000:0.0:1789760736.547849:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760736.551836:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495798144 00000100:00100000:3.0:1789760736.551851:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495798144 00000100:00100000:3.0:1789760736.551909:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720788075040 time 720831891810 state 0x0 ref 1 00000100:00100000:3.0:1789760736.551927:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720832152226 time 720831891810 state 0x1 ref 1 00000100:00100000:3.0:1789760736.552000:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720832152226 time 720872715974 state 0x0 ref 1 00000100:00100000:3.0:1789760736.552014:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495798144:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760736.552060:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22982656, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb007 00010000:00010000:3.0:1789760736.552069:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546139 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.552097:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760736.552102:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.552108:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546139 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.552122:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546139 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760736.583609:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972928 is committed 00002000:00010000:3.0:1789760736.583677:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22982656 -> 22986752 00010000:00010000:3.0:1789760736.583691:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546139 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.583704:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546139 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.583713:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546139 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.583721:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760736.583726:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546139 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.583735:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546139 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.583755:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546139 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.583763:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546139 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.583848:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495798144:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31831us (36002us total) trans 4294972928 rc 0/0 00000100:00100000:3.0:1789760736.583858:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5640 00000100:00100000:3.0:1789760736.583867:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720832152226 time 720872715974 state 0x2 ref 0 00000100:00100000:0.0:1789760736.589242:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760736.589688:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495798272 00000100:00100000:2.0:1789760736.589698:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495798272 00000100:00100000:2.0:1789760736.589753:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720832152226 time 720872715974 state 0x0 ref 1 00000100:00100000:2.0:1789760736.589768:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720872976390 time 720872715974 state 0x1 ref 1 00000100:00100000:2.0:1789760736.589801:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720872976390 time 720910517700 state 0x0 ref 1 00000100:00100000:2.0:1789760736.589813:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495798272:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760736.589861:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22986752, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb007 00010000:00010000:2.0:1789760736.589871:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546140 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.589885:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760736.589890:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.589896:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546140 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.589909:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546140 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760736.632191:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972929 is committed 00002000:00010000:2.0:1789760736.632421:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22986752 -> 22990848 00010000:00010000:2.0:1789760736.632447:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546140 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.632464:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546140 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.632473:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546140 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.632480:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760736.632485:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546140 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.632493:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546140 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.632512:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546140 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.632519:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546140 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.632613:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495798272:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42797us (43375us total) trans 4294972929 rc 0/0 00000100:00100000:2.0:1789760736.632624:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5641 00000100:00100000:2.0:1789760736.632634:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720872976390 time 720910517700 state 0x2 ref 0 00000100:00100000:2.0:1789760736.637387:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760736.637822:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495798400 00000100:00100000:1.0:1789760736.637832:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495798400 00000100:00100000:1.0:1789760736.637884:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720872976390 time 720910517700 state 0x0 ref 1 00000100:00100000:1.0:1789760736.637899:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720910778116 time 720910517700 state 0x1 ref 1 00000100:00100000:1.0:1789760736.637932:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720910778116 time 720958648729 state 0x0 ref 1 00000100:00100000:1.0:1789760736.637946:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495798400:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760736.637996:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22990848, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb007 00010000:00010000:1.0:1789760736.638005:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546147 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.638021:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760736.638025:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.638032:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546147 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.638046:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546147 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760736.666412:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972930 is committed 00002000:00010000:1.0:1789760736.666487:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22990848 -> 22994944 00002000:00010000:1.0:1789760736.666492:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760735 -> 1789760736 00002000:00010000:1.0:1789760736.666495:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760735 -> 1789760736 00010000:00010000:1.0:1789760736.666508:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546147 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.666522:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546147 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.666532:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546147 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.666540:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760736.666545:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546147 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.666554:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546147 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.666575:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546147 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.666584:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546147 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.666668:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495798400:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28720us (29285us total) trans 4294972930 rc 0/0 00000100:00100000:1.0:1789760736.666677:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5642 00000100:00100000:1.0:1789760736.666687:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720910778116 time 720958648729 state 0x2 ref 0 00000100:00100000:0.0:1789760736.673624:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760736.674253:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495798528 00000100:00100000:3.0:1789760736.674264:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495798528 00000100:00100000:3.0:1789760736.674373:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720910778116 time 720958648729 state 0x0 ref 1 00000100:00100000:3.0:1789760736.674398:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720958909145 time 720958648729 state 0x1 ref 1 00000100:00100000:3.0:1789760736.674481:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720958909145 time 720995196833 state 0x0 ref 1 00000100:00100000:3.0:1789760736.674494:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495798528:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760736.674539:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22994944, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb007 00010000:00010000:3.0:1789760736.674548:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454614e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.674562:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760736.674565:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.674572:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454614e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.674584:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454614e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760736.710132:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972931 is committed 00002000:00010000:0.0:1789760736.710332:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22994944 -> 22999040 00010000:00010000:0.0:1789760736.710348:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454614e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.710360:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454614e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.710374:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454614e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.710381:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760736.710386:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454614e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.710395:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454614e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.710416:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454614e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.710423:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454614e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.710521:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495798528:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36024us (36906us total) trans 4294972931 rc 0/0 00000100:00100000:0.0:1789760736.710531:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5643 00000100:00100000:0.0:1789760736.710540:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720958909145 time 720995196833 state 0x2 ref 0 00000100:00100000:1.0:1789760736.714533:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760736.714619:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495798656 00000100:00100000:2.0:1789760736.714628:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495798656 00000100:00100000:2.0:1789760736.714682:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720958909145 time 720995196833 state 0x0 ref 1 00000100:00100000:2.0:1789760736.714698:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720995457249 time 720995196833 state 0x1 ref 1 00000100:00100000:2.0:1789760736.714728:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720995457249 time 721035445199 state 0x0 ref 1 00000100:00100000:2.0:1789760736.714741:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495798656:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760736.714789:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22999040, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb007 00010000:00010000:2.0:1789760736.714798:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546155 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.714814:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760736.714818:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.714824:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546155 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.714837:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546155 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760736.756510:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972932 is committed 00002000:00010000:1.0:1789760736.757019:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22999040 -> 23003136 00010000:00010000:1.0:1789760736.757037:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546155 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.757050:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546155 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.757060:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546155 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.757068:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760736.757073:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546155 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.757091:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546155 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.757110:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546155 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.757119:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546155 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.758036:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495798656:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43289us (43506us total) trans 4294972932 rc 0/0 00000100:00100000:1.0:1789760736.758047:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5644 00000100:00100000:1.0:1789760736.758057:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720995457249 time 721035445199 state 0x2 ref 0 00000100:00100000:2.0:1789760736.765488:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760736.765566:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495798784 00000100:00100000:1.0:1789760736.765577:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495798784 00000100:00100000:1.0:1789760736.765636:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 720995457249 time 721035445199 state 0x0 ref 1 00000100:00100000:1.0:1789760736.765653:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721035705615 time 721035445199 state 0x1 ref 1 00000100:00100000:1.0:1789760736.765687:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721035705615 time 721086402937 state 0x0 ref 1 00000100:00100000:1.0:1789760736.765700:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495798784:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760736.765761:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23003136, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb007 00010000:00010000:1.0:1789760736.765772:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454615c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.765795:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760736.765799:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.765807:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454615c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.765822:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454615c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760736.811670:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972933 is committed 00002000:00010000:2.0:1789760736.811959:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23003136 -> 23007232 00010000:00010000:2.0:1789760736.811976:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454615c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.811992:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454615c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.812001:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454615c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.812009:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760736.812013:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454615c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.812023:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454615c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.812042:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454615c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.812049:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454615c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.812220:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495798784:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46518us (46738us total) trans 4294972933 rc 0/0 00000100:00100000:2.0:1789760736.812229:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5645 00000100:00100000:2.0:1789760736.812239:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721035705615 time 721086402937 state 0x2 ref 0 00000100:00100000:0.0:1789760736.816184:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760736.816748:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495798912 00000100:00100000:3.0:1789760736.816760:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495798912 00000100:00100000:3.0:1789760736.816813:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721035705615 time 721086402937 state 0x0 ref 1 00000100:00100000:3.0:1789760736.816830:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721086663353 time 721086402937 state 0x1 ref 1 00000100:00100000:3.0:1789760736.816898:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721086663353 time 721137614474 state 0x0 ref 1 00000100:00100000:3.0:1789760736.816911:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495798912:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760736.816960:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23007232, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb007 00010000:00010000:3.0:1789760736.816969:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546163 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.816990:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760736.816995:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.817001:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546163 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.817015:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546163 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760736.855378:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972934 is committed 00002000:00010000:3.0:1789760736.855477:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23007232 -> 23011328 00010000:00010000:3.0:1789760736.855493:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546163 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.855507:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546163 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.855516:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546163 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.855524:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760736.855528:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546163 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.855538:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546163 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.855559:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546163 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.855568:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546163 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.855661:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495798912:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38747us (39481us total) trans 4294972934 rc 0/0 00000100:00100000:3.0:1789760736.855672:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5646 00000100:00100000:3.0:1789760736.855681:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721086663353 time 721137614474 state 0x2 ref 0 00000100:00100000:1.0:1789760736.866417:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760736.866511:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495799040 00000100:00100000:2.0:1789760736.866520:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495799040 00000100:00100000:2.0:1789760736.866571:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721086663353 time 721137614474 state 0x0 ref 1 00000100:00100000:2.0:1789760736.866586:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721137874890 time 721137614474 state 0x1 ref 1 00000100:00100000:2.0:1789760736.866616:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721137874890 time 721187333620 state 0x0 ref 1 00000100:00100000:2.0:1789760736.866628:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495799040:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760736.866675:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23011328, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb007 00010000:00010000:2.0:1789760736.866684:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454616a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.866699:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760736.866704:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.866709:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454616a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.866722:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454616a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760736.908179:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23011328 -> 23015424 00010000:00010000:1.0:1789760736.908200:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454616a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.908214:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454616a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.908228:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454616a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.908237:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760736.908243:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454616a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.908253:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454616a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.908273:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454616a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.908298:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454616a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.908401:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495799040:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41768us (41986us total) trans 4294972935 rc 0/0 00000100:00100000:1.0:1789760736.908412:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5647 00000100:00100000:1.0:1789760736.908422:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721137874890 time 721187333620 state 0x2 ref 0 00000001:00080000:0.0:1789760736.908503:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972935 is committed 00000100:00100000:2.0:1789760736.912331:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760736.912435:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495799168 00000100:00100000:1.0:1789760736.912445:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495799168 00000100:00100000:1.0:1789760736.912498:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721137874890 time 721187333620 state 0x0 ref 1 00000100:00100000:1.0:1789760736.912512:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721187594036 time 721187333620 state 0x1 ref 1 00000100:00100000:1.0:1789760736.912535:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721187594036 time 721233251763 state 0x0 ref 1 00000100:00100000:1.0:1789760736.912547:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495799168:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760736.912593:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23015424, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb007 00010000:00010000:1.0:1789760736.912603:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546171 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.912618:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760736.912623:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.912629:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546171 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.912652:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546171 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760736.945726:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972936 is committed 00002000:00010000:3.0:1789760736.945831:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23015424 -> 23019520 00010000:00010000:3.0:1789760736.945857:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546171 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.945870:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546171 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.945879:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546171 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.945887:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760736.945892:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546171 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.945901:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546171 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.945926:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546171 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.945934:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546171 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.946026:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495799168:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33475us (33744us total) trans 4294972936 rc 0/0 00000100:00100000:3.0:1789760736.946037:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5648 00000100:00100000:3.0:1789760736.946047:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721187594036 time 721233251763 state 0x2 ref 0 00000100:00100000:2.0:1789760736.949818:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760736.949948:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495799296 00000100:00100000:3.0:1789760736.949957:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495799296 00000100:00100000:3.0:1789760736.950014:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721187594036 time 721233251763 state 0x0 ref 1 00000100:00100000:3.0:1789760736.950028:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721233512179 time 721233251763 state 0x1 ref 1 00000100:00100000:2.0:1789760736.950161:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721233512179 time 721270873932 state 0x0 ref 1 00000100:00100000:2.0:1789760736.950175:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495799296:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760736.950226:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23019520, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb007 00010000:00010000:2.0:1789760736.950236:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546178 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.950250:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760736.950255:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.950261:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546178 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.950274:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546178 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760736.990821:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972937 is committed 00002000:00010000:3.0:1789760736.991119:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23019520 -> 23023616 00010000:00010000:3.0:1789760736.991133:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546178 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.991158:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546178 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.991167:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546178 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.991181:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760736.991186:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546178 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.991195:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546178 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.991215:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546178 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.991223:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546178 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.991356:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495799296:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41178us (41543us total) trans 4294972937 rc 0/0 00000100:00100000:3.0:1789760736.991367:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5649 00000100:00100000:3.0:1789760736.991377:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721233512179 time 721270873932 state 0x2 ref 0 00000100:00100000:0.0:1789760736.996572:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760736.996662:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495799424 00000100:00100000:1.0:1789760736.996671:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495799424 00000100:00100000:1.0:1789760736.996728:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721233512179 time 721270873932 state 0x0 ref 1 00000100:00100000:1.0:1789760736.996743:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721271134348 time 721270873932 state 0x1 ref 1 00000100:00100000:1.0:1789760736.996772:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721271134348 time 721317488867 state 0x0 ref 1 00000100:00100000:1.0:1789760736.996785:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495799424:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760736.996905:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23023616, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb007 00010000:00010000:1.0:1789760736.996916:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454617f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.996933:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760736.996937:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.996944:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454617f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.996957:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454617f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760737.027227:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972938 is committed 00002000:00010000:3.0:1789760737.027439:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23023616 -> 23027712 00010000:00010000:3.0:1789760737.027455:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454617f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.027467:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454617f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.027482:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454617f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.027489:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760737.027494:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454617f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.027503:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454617f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.027523:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454617f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.027532:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454617f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.027631:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495799424:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30843us (31065us total) trans 4294972938 rc 0/0 00000100:00100000:3.0:1789760737.027641:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5650 00000100:00100000:3.0:1789760737.027655:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721271134348 time 721317488867 state 0x2 ref 0 00000100:00100000:0.0:1789760737.041799:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760737.041892:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495799552 00000100:00100000:3.0:1789760737.041903:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495799552 00000100:00100000:3.0:1789760737.041975:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721271134348 time 721317488867 state 0x0 ref 1 00000100:00100000:3.0:1789760737.041993:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721317749283 time 721317488867 state 0x1 ref 1 00000100:00100000:3.0:1789760737.042094:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721317749283 time 721362810278 state 0x0 ref 1 00000100:00100000:3.0:1789760737.042109:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495799552:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760737.042156:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23027712, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb007 00010000:00010000:3.0:1789760737.042165:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546186 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.042182:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760737.042187:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.042194:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546186 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.042209:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546186 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760737.042746:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789760737.076104:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23027712 -> 23031808 00010000:00010000:0.0:1789760737.076125:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546186 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.076139:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546186 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.076148:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546186 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.076157:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760737.076162:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546186 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.076171:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546186 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.076190:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546186 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.076198:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546186 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760737.076334:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495799552:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34223us (34542us total) trans 4294972939 rc 0/0 00000100:00100000:0.0:1789760737.076345:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5651 00000100:00100000:0.0:1789760737.076355:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721317749283 time 721362810278 state 0x2 ref 0 00000001:00080000:1.0:1789760737.076653:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972939 is committed 00000100:00100000:0.0:1789760737.084501:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760737.084589:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495799680 00000100:00100000:3.0:1789760737.084599:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495799680 00000100:00100000:3.0:1789760737.084654:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721317749283 time 721362810278 state 0x0 ref 1 00000100:00100000:3.0:1789760737.084670:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721363070694 time 721362810278 state 0x1 ref 1 00000100:00100000:3.0:1789760737.084698:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721363070694 time 721405415303 state 0x0 ref 1 00000100:00100000:3.0:1789760737.084711:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495799680:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760737.084758:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23031808, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb007 00010000:00010000:3.0:1789760737.084769:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454618d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.084786:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760737.084790:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.084796:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454618d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.084809:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454618d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.125361:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760737.128904:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972940 is committed 00002000:00010000:3.0:1789760737.129249:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23031808 -> 23035904 00010000:00010000:3.0:1789760737.129362:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454618d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.129377:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454618d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.129385:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454618d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.129392:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760737.129397:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454618d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.129407:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454618d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.129432:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454618d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.129440:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454618d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.129524:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495799680:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44810us (45027us total) trans 4294972940 rc 0/0 00000100:00100000:3.0:1789760737.129535:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5652 00000100:00100000:3.0:1789760737.129545:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721363070694 time 721405415303 state 0x2 ref 0 00000100:00100000:0.0:1789760737.135390:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760737.135520:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495799808 00000100:00100000:3.0:1789760737.135531:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495799808 00000100:00100000:3.0:1789760737.135587:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721363070694 time 721405415303 state 0x0 ref 1 00000100:00100000:3.0:1789760737.135603:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721405675719 time 721405415303 state 0x1 ref 1 00000100:00100000:3.0:1789760737.135633:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721405675719 time 721456349306 state 0x0 ref 1 00000100:00100000:3.0:1789760737.135646:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495799808:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760737.135697:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23035904, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb007 00010000:00010000:3.0:1789760737.135707:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546194 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.135724:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760737.135729:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.135736:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546194 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.135750:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546194 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760737.184240:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23035904 -> 23040000 00010000:00010000:3.0:1789760737.184259:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546194 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.184301:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546194 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.184310:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546194 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.184318:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760737.184323:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546194 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.184332:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546194 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.184352:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546194 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.184366:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546194 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.184784:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495799808:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49135us (49398us total) trans 4294972941 rc 0/0 00000100:00100000:3.0:1789760737.184797:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5653 00000100:00100000:3.0:1789760737.184806:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721405675719 time 721456349306 state 0x2 ref 0 00000001:00080000:1.0:1789760737.184962:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972941 is committed 00000100:00100000:0.0:1789760737.188406:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760737.188546:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495799936 00000100:00100000:3.0:1789760737.188555:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495799936 00000100:00100000:3.0:1789760737.188606:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721405675719 time 721456349306 state 0x0 ref 1 00000100:00100000:3.0:1789760737.188620:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721456609722 time 721456349306 state 0x1 ref 1 00000100:00100000:3.0:1789760737.188647:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721456609722 time 721509364891 state 0x0 ref 1 00000100:00100000:3.0:1789760737.188658:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495799936:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760737.188704:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23040000, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb007 00010000:00010000:3.0:1789760737.188711:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454619b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.188724:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760737.188728:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.188732:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454619b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.188747:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454619b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760737.239596:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972942 is committed 00002000:00010000:0.0:1789760737.239709:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23040000 -> 23044096 00010000:00010000:0.0:1789760737.239729:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454619b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.239744:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454619b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.239753:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454619b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.239760:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760737.239766:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454619b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.239776:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454619b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.239803:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454619b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.239811:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454619b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760737.239953:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495799936:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51289us (51548us total) trans 4294972942 rc 0/0 00000100:00100000:0.0:1789760737.239966:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5654 00000100:00100000:0.0:1789760737.239978:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721456609722 time 721509364891 state 0x2 ref 0 00000100:00100000:1.0:1789760737.245016:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760737.245249:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495800064 00000100:00100000:3.0:1789760737.245260:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495800064 00000100:00100000:3.0:1789760737.245528:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721456609722 time 721509364891 state 0x0 ref 1 00000100:00100000:3.0:1789760737.245544:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721509625307 time 721509364891 state 0x1 ref 1 00000100:00100000:3.0:1789760737.245610:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721509625307 time 721566326105 state 0x0 ref 1 00000100:00100000:3.0:1789760737.245625:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495800064:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760737.245674:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23044096, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb007 00010000:00010000:3.0:1789760737.245684:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545461a2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.245700:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760737.245704:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.245710:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545461a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.245723:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545461a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760737.287404:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23044096 -> 23048192 00010000:00010000:3.0:1789760737.287424:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545461a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.287437:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545461a2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.287447:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545461a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.287455:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760737.287461:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545461a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.287470:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545461a2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.287491:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545461a2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.287498:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545461a2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.287594:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495800064:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41967us (42583us total) trans 4294972943 rc 0/0 00000100:00100000:3.0:1789760737.287605:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5655 00000100:00100000:3.0:1789760737.287616:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721509625307 time 721566326105 state 0x2 ref 0 00000100:00100000:3.0:1789760737.296200:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760737.296376:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495800192 00000100:00100000:0.0:1789760737.296386:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495800192 00000100:00100000:0.0:1789760737.296449:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721509625307 time 721566326105 state 0x0 ref 1 00000100:00100000:0.0:1789760737.296465:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721566586521 time 721566326105 state 0x1 ref 1 00000100:00100000:0.0:1789760737.296505:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721566586521 time 721617220488 state 0x0 ref 1 00000100:00100000:0.0:1789760737.296518:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04930a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495800192:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760737.296568:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23048192, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb007 00010000:00010000:0.0:1789760737.296578:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545461a9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.296595:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760737.296601:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760737.296608:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545461a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.296623:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545461a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760737.296949:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972943 is committed 00002000:00010000:1.0:1789760737.326872:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23048192 -> 23052288 00010000:00010000:1.0:1789760737.326889:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545461a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.326902:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545461a9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.326911:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545461a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.326926:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760737.326931:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545461a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.326939:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545461a9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.326957:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545461a9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.326964:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545461a9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760737.327049:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04930a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495800192:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30528us (30854us total) trans 4294972944 rc 0/0 00000100:00100000:1.0:1789760737.327059:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5656 00000100:00100000:1.0:1789760737.327068:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721566586521 time 721617220488 state 0x2 ref 0 00000001:00080000:2.0:1789760737.328053:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972944 is committed 00000100:00100000:0.0:1789760737.334849:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760737.334923:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495800320 00000100:00100000:3.0:1789760737.334934:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495800320 00000100:00100000:3.0:1789760737.334985:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721566586521 time 721617220488 state 0x0 ref 1 00000100:00100000:3.0:1789760737.335000:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721617480904 time 721617220488 state 0x1 ref 1 00000100:00100000:3.0:1789760737.335033:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721617480904 time 721655749820 state 0x0 ref 1 00000100:00100000:3.0:1789760737.335046:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495800320:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760737.335112:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23052288, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb007 00010000:00010000:3.0:1789760737.335122:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545461b0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.335137:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760737.335140:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.335147:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545461b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.335175:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545461b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760737.382971:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972945 is committed 00002000:00010000:0.0:1789760737.383224:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23052288 -> 23056384 00010000:00010000:0.0:1789760737.383240:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545461b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.383253:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545461b0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.383263:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545461b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.383271:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760737.383276:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545461b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.383320:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545461b0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.383340:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545461b0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.383353:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545461b0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760737.383440:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495800320:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48390us (48595us total) trans 4294972945 rc 0/0 00000100:00100000:0.0:1789760737.383450:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5657 00000100:00100000:0.0:1789760737.383460:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721617480904 time 721655749820 state 0x2 ref 0 00000100:00100000:0.0:1789760737.390495:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760737.390714:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495800448 00000100:00100000:3.0:1789760737.390923:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495800448 00000100:00100000:3.0:1789760737.390979:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721617480904 time 721655749820 state 0x0 ref 1 00000100:00100000:3.0:1789760737.390994:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721656010236 time 721655749820 state 0x1 ref 1 00000100:00100000:3.0:1789760737.391096:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721656010236 time 721711812068 state 0x0 ref 1 00000100:00100000:3.0:1789760737.391108:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495800448:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760737.391158:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23056384, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb007 00010000:00010000:3.0:1789760737.391167:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461b7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.391182:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760737.391186:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.391193:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.391206:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760737.431131:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23056384 -> 23060480 00010000:00010000:3.0:1789760737.431151:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.431165:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461b7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.431175:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.431183:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760737.431189:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.431198:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461b7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.431220:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461b7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.431227:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461b7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760737.431352:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972946 is committed 00000100:00100000:3.0:1789760737.431434:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495800448:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40322us (40944us total) trans 4294972946 rc 0/0 00000100:00100000:3.0:1789760737.431445:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5658 00000100:00100000:3.0:1789760737.431455:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721656010236 time 721711812068 state 0x2 ref 0 00000100:00100000:3.0:1789760737.437645:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760737.437757:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495800576 00000100:00100000:1.0:1789760737.437767:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495800576 00000100:00100000:1.0:1789760737.437824:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721656010236 time 721711812068 state 0x0 ref 1 00000100:00100000:1.0:1789760737.437839:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721712072484 time 721711812068 state 0x1 ref 1 00000100:00100000:1.0:1789760737.437875:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721712072484 time 721758591493 state 0x0 ref 1 00000100:00100000:1.0:1789760737.437892:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04931180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495800576:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760737.437940:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23060480, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb007 00010000:00010000:1.0:1789760737.437949:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461be lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.437965:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760737.437969:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760737.437976:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.437989:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760737.490029:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972947 is committed 00002000:00010000:1.0:1789760737.490886:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23060480 -> 23064576 00010000:00010000:1.0:1789760737.490902:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.490915:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461be lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.490924:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.490932:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760737.490936:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.490945:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461be lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.490965:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461be lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.490972:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461be lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760737.491120:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495800576:12345-192.168.203.9@tcp:4:dd.0 Request processed in 53225us (53478us total) trans 4294972947 rc 0/0 00000100:00100000:1.0:1789760737.491131:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5659 00000100:00100000:1.0:1789760737.491141:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721712072484 time 721758591493 state 0x2 ref 0 00000100:00100000:2.0:1789760737.497357:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760737.497481:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495800704 00000100:00100000:0.0:1789760737.497490:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495800704 00000100:00100000:0.0:1789760737.497544:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721712072484 time 721758591493 state 0x0 ref 1 00000100:00100000:0.0:1789760737.497559:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721758851909 time 721758591493 state 0x1 ref 1 00000100:00100000:0.0:1789760737.497587:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721758851909 time 721818303760 state 0x0 ref 1 00000100:00100000:0.0:1789760737.497600:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495800704:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760737.497658:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23064576, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb007 00010000:00010000:0.0:1789760737.497668:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545461c5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.497685:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760737.497690:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760737.497697:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545461c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.497713:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545461c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760737.531662:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972948 is committed 00002000:00010000:1.0:1789760737.531779:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23064576 -> 23068672 00010000:00010000:1.0:1789760737.531795:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545461c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.531808:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545461c5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.531817:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545461c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.531825:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760737.531841:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545461c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.531851:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545461c5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.531872:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545461c5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.531880:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545461c5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760737.531972:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495800704:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34370us (34620us total) trans 4294972948 rc 0/0 00000100:00100000:1.0:1789760737.531982:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5660 00000100:00100000:1.0:1789760737.531992:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721758851909 time 721818303760 state 0x2 ref 0 00000100:00100000:0.0:1789760737.539383:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760737.539800:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495800832 00000100:00100000:3.0:1789760737.539810:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495800832 00000100:00100000:3.0:1789760737.539872:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721758851909 time 721818303760 state 0x0 ref 1 00000100:00100000:3.0:1789760737.539886:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721818564176 time 721818303760 state 0x1 ref 1 00000100:00100000:3.0:1789760737.539918:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721818564176 time 721860634843 state 0x0 ref 1 00000100:00100000:3.0:1789760737.539933:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f921c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495800832:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760737.539981:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23068672, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb007 00010000:00010000:3.0:1789760737.539991:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461cc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.540007:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760737.540011:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.540018:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.540031:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760737.566523:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f0492d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520306304:0@lo:400:kworker.0 00000100:00100000:3.0:1789760737.566533:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f0492ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3318:1876699520306176:0@lo:400:kworker.0 00000100:00100000:1.0:1789760737.566592:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760737.566663:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760737.566672:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9290404a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520306432:0@lo:400:kworker.0 00000100:00100000:1.0:1789760737.566708:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760737.566749:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520306304 00000100:00100000:1.0:1789760737.566757:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520306304 00000100:00100000:3.0:1789760737.566792:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760737.566817:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9290404a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3318:1876699520306688:0@lo:400:kworker.0 00000100:00100000:1.0:1789760737.566818:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520306176 00000100:00100000:1.0:1789760737.566823:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520306176 00000100:00100000:1.0:1789760737.566852:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 477 00000100:00100000:3.0:1789760737.566854:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760737.566863:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520306304:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760737.566877:0:3318:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1789760737.566885:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f0492e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:59c4e264-9b75-482b-8813-3d1dc124f384:3318:1876699520306048:0@lo:400:kworker.0 00000100:00100000:1.0:1789760737.566931:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520306304:12345-0@lo:400:kworker.0 Request processed in 66us (335us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760737.566931:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760737.566942:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 477 00000100:00100000:1.0:1789760737.566957:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 478 00000100:00100000:1.0:1789760737.566962:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04932300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3318:x1876699520306176:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760737.567035:0:13058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520306432 00000100:00100000:2.0:1789760737.567040:0:13058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520306432 00000100:00080000:0.0:1789760737.567064:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760737.567069:0:13058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520306688 00000100:00080000:0.0:1789760737.567079:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760737.567085:0:13058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520306688 00000100:00080000:0.0:1789760737.567111:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789760737.567177:0:13058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 222 00000100:00100000:2.0:1789760737.567188:0:13058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520306432:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760737.567246:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04932300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3318:x1876699520306176:12345-0@lo:400:kworker.0 Request processed in 283us (454us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760737.567252:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 478 00000100:00100000:1.0:1789760737.567271:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 223 00000100:00100000:1.0:1789760737.567276:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04933b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3318:x1876699520306688:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760737.567324:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04933b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3318:x1876699520306688:12345-0@lo:400:kworker.0 Request processed in 47us (473us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760737.567330:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 223 00000100:00100000:3.0:1789760737.567348:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9290404a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3318:1876699520306560:0@lo:400:kworker.0 00000100:00100000:1.0:1789760737.567376:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f0492d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520306304:0@lo:400:kworker.0 00000100:00100000:3.0:1789760737.567384:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760737.567423:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f0492ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3318:1876699520306176:0@lo:400:kworker.0 00000100:00100000:3.0:1789760737.567433:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9290404a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3318:1876699520306688:0@lo:400:kworker.0 00000100:00100000:3.0:1789760737.567461:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520306560 00000100:00100000:3.0:1789760737.567465:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520306560 00000100:00100000:3.0:1789760737.567504:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 479 00000100:00100000:3.0:1789760737.567511:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04931c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1876699520306560:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760737.567670:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1876699520306560:12345-0@lo:400:kworker.0 Request processed in 158us (288us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760737.567677:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 479 00000100:00100000:3.0:1789760737.567706:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9290404a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3318:1876699520306560:0@lo:400:kworker.0 00000001:00080000:3.0:1789760737.569735:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972949 is committed 00002000:00010000:3.0:1789760737.569804:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23068672 -> 23072768 00002000:00010000:3.0:1789760737.569809:0:6714:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 45063 -> 47111 00010000:00010000:3.0:1789760737.569821:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.569833:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461cc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.569842:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.569849:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760737.569854:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.569864:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461cc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.569889:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461cc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.569898:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461cc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.569986:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f921c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495800832:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30051us (30609us total) trans 4294972949 rc 0/0 00000100:00100000:3.0:1789760737.569997:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5661 00000100:00100000:3.0:1789760737.570006:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721818564176 time 721860634843 state 0x2 ref 0 00000100:00100000:3.0:1789760737.570480:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520306048 00000100:00100000:3.0:1789760737.570489:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520306048 00000100:00100000:3.0:1789760737.570526:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 399 00000100:00100000:3.0:1789760737.570534:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929012311850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:59c4e264-9b75-482b-8813-3d1dc124f384+19:3318:x1876699520306048:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760737.570604:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929012311850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:59c4e264-9b75-482b-8813-3d1dc124f384+19:3318:x1876699520306048:12345-0@lo:400:kworker.0 Request processed in 70us (3676us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760737.570610:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 399 00000100:00100000:3.0:1789760737.570657:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f0492e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:59c4e264-9b75-482b-8813-3d1dc124f384:3318:1876699520306048:0@lo:400:kworker.0 00000100:00080000:1.0:1789760737.571260:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760737.571434:0:13058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520306432:12345-0@lo:400:kworker.0 Request processed in 4247us (4724us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760737.571445:0:13058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 222 00000100:00100000:1.0:1789760737.572223:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9290404a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520306432:0@lo:400:kworker.0 00000100:00100000:3.0:1789760737.576867:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760737.577021:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495800960 00000100:00100000:1.0:1789760737.577030:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495800960 00000100:00100000:1.0:1789760737.577088:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721818564176 time 721860634843 state 0x0 ref 1 00000100:00100000:1.0:1789760737.577110:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721860895259 time 721860634843 state 0x1 ref 1 00000100:00100000:1.0:1789760737.577252:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721860895259 time 721897966740 state 0x0 ref 1 00000100:00100000:1.0:1789760737.577266:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04931f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495800960:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760737.577351:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23072768, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb807 00010000:00010000:1.0:1789760737.577361:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461d3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.577377:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760737.577381:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760737.577392:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.577407:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760737.632996:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972950 is committed 00002000:00010000:1.0:1789760737.633152:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23072768 -> 23076864 00010000:00010000:1.0:1789760737.633168:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.633180:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461d3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.633191:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.633198:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760737.633203:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.633213:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461d3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.633233:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461d3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.633241:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461d3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760737.633441:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495800960:12345-192.168.203.9@tcp:4:dd.0 Request processed in 56172us (56578us total) trans 4294972950 rc 0/0 00000100:00100000:1.0:1789760737.633450:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5662 00000100:00100000:1.0:1789760737.633459:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721860895259 time 721897966740 state 0x2 ref 0 00000100:00100000:2.0:1789760737.636956:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760737.637179:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495801088 00000100:00100000:1.0:1789760737.637189:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495801088 00000100:00100000:1.0:1789760737.637243:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721860895259 time 721897966740 state 0x0 ref 1 00000100:00100000:1.0:1789760737.637257:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721898227156 time 721897966740 state 0x1 ref 1 00000100:00100000:1.0:1789760737.637325:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721898227156 time 721958041225 state 0x0 ref 1 00000100:00100000:1.0:1789760737.637338:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495801088:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760737.637384:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23076864, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb807 00010000:00010000:1.0:1789760737.637406:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545461da lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.637424:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760737.637429:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760737.637436:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545461da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.637449:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545461da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760737.672731:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972951 is committed 00002000:00010000:3.0:1789760737.672878:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23076864 -> 23080960 00002000:00010000:3.0:1789760737.672883:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760736 -> 1789760737 00002000:00010000:3.0:1789760737.672885:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760736 -> 1789760737 00010000:00010000:3.0:1789760737.672897:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545461da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.672912:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545461da lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.672922:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545461da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.672930:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760737.672936:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545461da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.672944:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545461da lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.672964:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545461da lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.672971:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545461da lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.673067:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495801088:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35725us (36114us total) trans 4294972951 rc 0/0 00000100:00100000:3.0:1789760737.673088:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5663 00000100:00100000:3.0:1789760737.673098:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721898227156 time 721958041225 state 0x2 ref 0 00000100:00100000:2.0:1789760737.678763:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760737.678851:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495801216 00000100:00100000:3.0:1789760737.678860:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495801216 00000100:00100000:3.0:1789760737.678909:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721898227156 time 721958041225 state 0x0 ref 1 00000100:00100000:3.0:1789760737.678925:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721958301641 time 721958041225 state 0x1 ref 1 00000100:00100000:3.0:1789760737.678982:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721958301641 time 721999698473 state 0x0 ref 1 00000100:00100000:3.0:1789760737.678994:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04292a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495801216:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760737.679041:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23080960, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:3.0:1789760737.679059:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461e1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.679084:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760737.679088:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.679095:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.679109:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760737.739626:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23080960 -> 23085056 00010000:00010000:3.0:1789760737.739645:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.739659:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461e1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.739673:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.739681:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760737.739685:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.739694:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461e1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.739716:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461e1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.739724:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461e1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.739820:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04292a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495801216:12345-192.168.203.9@tcp:4:dd.0 Request processed in 60822us (61061us total) trans 4294972952 rc 0/0 00000100:00100000:3.0:1789760737.739832:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5664 00000100:00100000:3.0:1789760737.739842:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721958301641 time 721999698473 state 0x2 ref 0 00000001:00080000:1.0:1789760737.740031:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972952 is committed 00000100:00100000:0.0:1789760737.745243:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760737.745416:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495801344 00000100:00100000:1.0:1789760737.745426:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495801344 00000100:00100000:1.0:1789760737.745499:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721958301641 time 721999698473 state 0x0 ref 1 00000100:00100000:1.0:1789760737.745514:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721999958889 time 721999698473 state 0x1 ref 1 00000100:00100000:1.0:1789760737.745547:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721999958889 time 722066263593 state 0x0 ref 1 00000100:00100000:1.0:1789760737.745559:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f923100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495801344:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760737.745608:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23085056, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:1.0:1789760737.745617:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461e8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.745631:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760737.745635:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760737.745641:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.745655:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760737.780038:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972953 is committed 00002000:00010000:3.0:1789760737.780548:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23085056 -> 23089152 00010000:00010000:3.0:1789760737.780562:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.780579:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461e8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.780587:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.780594:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760737.780599:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.780607:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461e8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.780626:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461e8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.780633:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545461e8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.780755:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f923100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495801344:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35193us (36626us total) trans 4294972953 rc 0/0 00000100:00100000:3.0:1789760737.780766:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5665 00000100:00100000:3.0:1789760737.780777:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721999958889 time 722066263593 state 0x2 ref 0 00000100:00100000:3.0:1789760737.786854:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760737.787006:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495801472 00000100:00100000:1.0:1789760737.787016:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495801472 00000100:00100000:1.0:1789760737.787093:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 721999958889 time 722066263593 state 0x0 ref 1 00000100:00100000:1.0:1789760737.787108:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722066524009 time 722066263593 state 0x1 ref 1 00000100:00100000:1.0:1789760737.787149:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722066524009 time 722107865316 state 0x0 ref 1 00000100:00100000:1.0:1789760737.787162:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495801472:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760737.787213:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23089152, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:1.0:1789760737.787222:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545461ef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.787239:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760737.787242:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760737.787248:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545461ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.787260:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545461ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760737.820957:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23089152 -> 23093248 00010000:00010000:3.0:1789760737.820978:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545461ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.820993:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545461ef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.821002:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545461ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.821009:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760737.821014:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545461ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.821024:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545461ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.821055:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545461ef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.821063:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545461ef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.821185:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495801472:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34021us (34335us total) trans 4294972954 rc 0/0 00000100:00100000:3.0:1789760737.821196:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5666 00000100:00100000:3.0:1789760737.821206:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722066524009 time 722107865316 state 0x2 ref 0 00000001:00080000:0.0:1789760737.822033:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972954 is committed 00000100:00100000:0.0:1789760737.829475:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760737.829702:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495801600 00000100:00100000:3.0:1789760737.829712:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495801600 00000100:00100000:3.0:1789760737.829764:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722066524009 time 722107865316 state 0x0 ref 1 00000100:00100000:3.0:1789760737.829780:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722108125732 time 722107865316 state 0x1 ref 1 00000100:00100000:3.0:1789760737.829854:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722108125732 time 722150570691 state 0x0 ref 1 00000100:00100000:3.0:1789760737.829867:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f922300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495801600:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760737.829914:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23093248, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:3.0:1789760737.829922:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461f6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.829937:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760737.829941:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.829947:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.829959:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760737.874467:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23093248 -> 23097344 00010000:00010000:3.0:1789760737.874496:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.874509:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461f6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.874519:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.874526:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760737.874530:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.874540:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461f6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.874560:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461f6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.874568:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461f6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.874665:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f922300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495801600:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44794us (45191us total) trans 4294972955 rc 0/0 00000100:00100000:3.0:1789760737.874676:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5667 00000100:00100000:3.0:1789760737.874686:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722108125732 time 722150570691 state 0x2 ref 0 00000001:00080000:1.0:1789760737.878413:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972955 is committed 00000100:00100000:2.0:1789760737.885506:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760737.885650:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495801728 00000100:00100000:3.0:1789760737.885660:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495801728 00000100:00100000:3.0:1789760737.885708:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722108125732 time 722150570691 state 0x0 ref 1 00000100:00100000:3.0:1789760737.885728:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722150831107 time 722150570691 state 0x1 ref 1 00000100:00100000:3.0:1789760737.885797:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722150831107 time 722206513317 state 0x0 ref 1 00000100:00100000:3.0:1789760737.885810:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903639f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495801728:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760737.885859:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23097344, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:3.0:1789760737.885868:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461fd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.885885:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760737.885890:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.885897:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.885911:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760737.936657:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972956 is committed 00002000:00010000:3.0:1789760737.936839:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23097344 -> 23101440 00010000:00010000:3.0:1789760737.936854:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.936867:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461fd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.936877:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.936884:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760737.936890:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.936899:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461fd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.936919:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461fd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.936927:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545461fd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.937025:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903639f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495801728:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51212us (51525us total) trans 4294972956 rc 0/0 00000100:00100000:3.0:1789760737.937035:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5668 00000100:00100000:3.0:1789760737.937045:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722150831107 time 722206513317 state 0x2 ref 0 00000100:00100000:2.0:1789760737.946654:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760737.946741:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495801856 00000100:00100000:3.0:1789760737.946751:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495801856 00000100:00100000:3.0:1789760737.946809:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722150831107 time 722206513317 state 0x0 ref 1 00000100:00100000:3.0:1789760737.946826:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722206773733 time 722206513317 state 0x1 ref 1 00000100:00100000:3.0:1789760737.946856:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722206773733 time 722267572645 state 0x0 ref 1 00000100:00100000:3.0:1789760737.946868:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04290700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495801856:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760737.946915:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23101440, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:3.0:1789760737.946924:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546204 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.946939:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760737.946944:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.946950:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546204 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.946963:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546204 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760737.979539:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972957 is committed 00002000:00010000:3.0:1789760737.979696:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23101440 -> 23105536 00010000:00010000:3.0:1789760737.979711:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546204 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.979724:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546204 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.979733:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546204 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.979740:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760737.979745:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546204 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.979755:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546204 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.979780:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546204 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.979787:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546204 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.979885:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04290700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495801856:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33014us (33235us total) trans 4294972957 rc 0/0 00000100:00100000:3.0:1789760737.979895:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5669 00000100:00100000:3.0:1789760737.979905:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722206773733 time 722267572645 state 0x2 ref 0 00000100:00100000:1.0:1789760737.985972:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760737.986176:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495801984 00000100:00100000:3.0:1789760737.986186:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495801984 00000100:00100000:3.0:1789760737.986239:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722206773733 time 722267572645 state 0x0 ref 1 00000100:00100000:3.0:1789760737.986256:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722267833061 time 722267572645 state 0x1 ref 1 00000100:00100000:2.0:1789760737.986385:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722267833061 time 722307098634 state 0x0 ref 1 00000100:00100000:2.0:1789760737.986400:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495801984:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760737.986447:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23105536, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:2.0:1789760737.986457:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454620b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.986474:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760737.986478:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.986484:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454620b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.986498:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454620b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760738.032401:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972958 is committed 00002000:00010000:0.0:1789760738.032516:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23105536 -> 23109632 00010000:00010000:0.0:1789760738.032531:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454620b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.032544:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454620b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.032553:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454620b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.032560:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760738.032564:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454620b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.032596:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454620b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.032615:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454620b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.032623:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454620b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.032712:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495801984:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46311us (46746us total) trans 4294972958 rc 0/0 00000100:00100000:0.0:1789760738.032723:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5670 00000100:00100000:0.0:1789760738.032733:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722267833061 time 722307098634 state 0x2 ref 0 00000100:00100000:3.0:1789760738.043329:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760738.043542:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495802112 00000100:00100000:1.0:1789760738.043551:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495802112 00000100:00100000:1.0:1789760738.043605:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722267833061 time 722307098634 state 0x0 ref 1 00000100:00100000:1.0:1789760738.043621:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722307359050 time 722307098634 state 0x1 ref 1 00000100:00100000:1.0:1789760738.043651:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722307359050 time 722364367150 state 0x0 ref 1 00000100:00100000:1.0:1789760738.043664:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495802112:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760738.043713:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23109632, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:1.0:1789760738.043722:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546212 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.043738:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760738.043742:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.043749:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546212 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.043762:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546212 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.078493:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9290404a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520306816:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760738.078562:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760738.078654:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520306816 00000100:00100000:3.0:1789760738.078662:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520306816 00000100:00100000:3.0:1789760738.078703:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 198 00000100:00100000:3.0:1789760738.078711:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520306816:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760738.078743:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760738.078786:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520306816:12345-0@lo:13:osp-pre-1-0.0 Request processed in 75us (224us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760738.078794:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 198 00000100:00100000:3.0:1789760738.079518:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9290404a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520306816:0@lo:13:osp-pre-1-0.0 00000001:00080000:3.0:1789760738.083561:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972959 is committed 00002000:00010000:0.0:1789760738.084201:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23109632 -> 23113728 00010000:00010000:0.0:1789760738.084217:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546212 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.084231:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546212 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.084240:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546212 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.084247:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760738.084254:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546212 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.084263:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546212 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.084307:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546212 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.084315:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546212 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.084398:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495802112:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40732us (41112us total) trans 4294972959 rc 0/0 00000100:00100000:0.0:1789760738.084409:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5671 00000100:00100000:0.0:1789760738.084418:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722307359050 time 722364367150 state 0x2 ref 0 00000100:00100000:3.0:1789760738.090524:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760738.090942:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495802240 00000100:00100000:2.0:1789760738.090953:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495802240 00000100:00100000:2.0:1789760738.091004:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722307359050 time 722364367150 state 0x0 ref 1 00000100:00100000:2.0:1789760738.091020:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722364627566 time 722364367150 state 0x1 ref 1 00000100:00100000:2.0:1789760738.091054:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722364627566 time 722411770314 state 0x0 ref 1 00000100:00100000:2.0:1789760738.091078:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495802240:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760738.091127:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23113728, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:2.0:1789760738.091136:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546219 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.091153:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760738.091158:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.091164:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546219 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.091178:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546219 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760738.123725:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23113728 -> 23117824 00010000:00010000:2.0:1789760738.123745:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546219 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.123758:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546219 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.123769:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546219 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.123776:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760738.123781:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546219 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.123791:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546219 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.123811:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546219 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.123819:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546219 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.124199:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495802240:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33129us (33679us total) trans 4294972960 rc 0/0 00000100:00100000:2.0:1789760738.124211:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5672 00000100:00100000:2.0:1789760738.124222:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722364627566 time 722411770314 state 0x2 ref 0 00000001:00080000:3.0:1789760738.124330:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972960 is committed 00000100:00100000:1.0:1789760738.127838:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760738.127932:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495802368 00000100:00100000:0.0:1789760738.127942:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495802368 00000100:00100000:0.0:1789760738.127997:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722364627566 time 722411770314 state 0x0 ref 1 00000100:00100000:0.0:1789760738.128024:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722412030730 time 722411770314 state 0x1 ref 1 00000100:00100000:0.0:1789760738.128056:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722412030730 time 722448772630 state 0x0 ref 1 00000100:00100000:0.0:1789760738.128078:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495802368:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760738.128127:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23117824, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:0.0:1789760738.128138:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546220 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.128154:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760738.128158:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.128164:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546220 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.128177:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546220 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.142789:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760738.156550:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972961 is committed 00002000:00010000:3.0:1789760738.156659:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23117824 -> 23121920 00010000:00010000:3.0:1789760738.156675:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546220 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.156688:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546220 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.156696:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546220 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.156704:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760738.156709:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546220 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.156719:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546220 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.156739:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546220 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.156753:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546220 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.156929:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495802368:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28857us (29103us total) trans 4294972961 rc 0/0 00000100:00100000:3.0:1789760738.156945:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5673 00000100:00100000:3.0:1789760738.156955:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722412030730 time 722448772630 state 0x2 ref 0 00000100:00100000:0.0:1789760738.160455:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760738.160739:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495802496 00000100:00100000:1.0:1789760738.160749:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495802496 00000100:00100000:1.0:1789760738.160804:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722412030730 time 722448772630 state 0x0 ref 1 00000100:00100000:1.0:1789760738.160818:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722449033046 time 722448772630 state 0x1 ref 1 00000100:00100000:3.0:1789760738.161140:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722449033046 time 722481854076 state 0x0 ref 1 00000100:00100000:3.0:1789760738.161156:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f921c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495802496:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760738.161204:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23121920, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:3.0:1789760738.161213:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546227 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.161230:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760738.161234:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.161240:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546227 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.161255:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546227 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760738.187577:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972962 is committed 00002000:00010000:1.0:1789760738.187735:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23121920 -> 23126016 00010000:00010000:1.0:1789760738.187750:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546227 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.187763:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546227 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.187773:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546227 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.187781:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760738.187786:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546227 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.187794:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546227 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.187812:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546227 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.187821:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546227 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760738.187911:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f921c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495802496:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26754us (27460us total) trans 4294972962 rc 0/0 00000100:00100000:1.0:1789760738.187922:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5674 00000100:00100000:1.0:1789760738.187932:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722449033046 time 722481854076 state 0x2 ref 0 00000100:00100000:2.0:1789760738.194628:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760738.194950:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495802624 00000100:00100000:0.0:1789760738.194959:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495802624 00000100:00100000:0.0:1789760738.195008:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722449033046 time 722481854076 state 0x0 ref 1 00000100:00100000:0.0:1789760738.195023:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722482114492 time 722481854076 state 0x1 ref 1 00000100:00100000:0.0:1789760738.195057:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722482114492 time 722515773863 state 0x0 ref 1 00000100:00100000:0.0:1789760738.195078:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495802624:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760738.195121:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23126016, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:0.0:1789760738.195129:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454622e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.195149:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760738.195159:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.195166:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454622e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.195179:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454622e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760738.237727:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972963 is committed 00002000:00010000:0.0:1789760738.237895:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23126016 -> 23130112 00010000:00010000:0.0:1789760738.237911:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454622e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.237924:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454622e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.237933:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454622e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.237941:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760738.237946:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454622e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.237955:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454622e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.237975:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454622e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.237983:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454622e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.238257:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495802624:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43185us (43633us total) trans 4294972963 rc 0/0 00000100:00100000:0.0:1789760738.238268:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5675 00000100:00100000:0.0:1789760738.238311:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722482114492 time 722515773863 state 0x2 ref 0 00000100:00100000:2.0:1789760738.241598:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760738.241684:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495802752 00000100:00100000:3.0:1789760738.241695:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495802752 00000100:00100000:3.0:1789760738.241749:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722482114492 time 722515773863 state 0x0 ref 1 00000100:00100000:3.0:1789760738.241765:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722516034279 time 722515773863 state 0x1 ref 1 00000100:00100000:3.0:1789760738.241797:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722516034279 time 722562513440 state 0x0 ref 1 00000100:00100000:3.0:1789760738.241809:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04291180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495802752:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760738.241969:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23130112, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:3.0:1789760738.241978:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546235 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.241994:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760738.241998:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.242005:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546235 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.242019:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546235 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760738.293238:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972964 is committed 00002000:00010000:1.0:1789760738.293402:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23130112 -> 23134208 00010000:00010000:1.0:1789760738.293429:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546235 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.293450:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546235 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.293459:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546235 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.293466:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760738.293471:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546235 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.293480:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546235 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.293498:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546235 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.293504:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546235 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760738.293593:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04291180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495802752:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51781us (52000us total) trans 4294972964 rc 0/0 00000100:00100000:1.0:1789760738.293603:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5676 00000100:00100000:1.0:1789760738.293612:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722516034279 time 722562513440 state 0x2 ref 0 00000100:00100000:1.0:1789760738.298147:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760738.298420:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495802880 00000100:00100000:3.0:1789760738.298431:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495802880 00000100:00100000:3.0:1789760738.298484:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722516034279 time 722562513440 state 0x0 ref 1 00000100:00100000:3.0:1789760738.298500:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722562773856 time 722562513440 state 0x1 ref 1 00000100:00100000:3.0:1789760738.298571:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722562773856 time 722619286812 state 0x0 ref 1 00000100:00100000:3.0:1789760738.298587:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495802880:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760738.298640:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23134208, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:3.0:1789760738.298650:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454623c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.298667:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760738.298674:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.298680:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454623c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.298695:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454623c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760738.331520:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23134208 -> 23138304 00010000:00010000:3.0:1789760738.331537:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454623c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.331550:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454623c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.331559:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454623c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.331567:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760738.331572:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454623c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.331582:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454623c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.331599:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454623c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.331606:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454623c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.331691:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495802880:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33102us (33549us total) trans 4294972965 rc 0/0 00000100:00100000:3.0:1789760738.331701:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5677 00000100:00100000:3.0:1789760738.331711:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722562773856 time 722619286812 state 0x2 ref 0 00000001:00080000:1.0:1789760738.331781:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972965 is committed 00000100:00100000:2.0:1789760738.336178:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760738.336416:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495803008 00000100:00100000:0.0:1789760738.336426:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495803008 00000100:00100000:0.0:1789760738.336478:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722562773856 time 722619286812 state 0x0 ref 1 00000100:00100000:0.0:1789760738.336493:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722619547228 time 722619286812 state 0x1 ref 1 00000100:00100000:1.0:1789760738.336647:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722619547228 time 722657360637 state 0x0 ref 1 00000100:00100000:1.0:1789760738.336661:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495803008:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760738.336708:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23138304, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:1.0:1789760738.336717:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546243 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.336733:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760738.336737:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.336743:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546243 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.336756:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546243 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760738.358431:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972966 is committed 00002000:00010000:2.0:1789760738.358546:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23138304 -> 23142400 00010000:00010000:2.0:1789760738.358562:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546243 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.358574:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546243 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.358583:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546243 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.358590:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760738.358595:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546243 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.358603:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546243 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.358628:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546243 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.358635:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546243 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.358978:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495803008:12345-192.168.203.9@tcp:4:dd.0 Request processed in 22314us (22804us total) trans 4294972966 rc 0/0 00000100:00100000:2.0:1789760738.358989:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5678 00000100:00100000:2.0:1789760738.358998:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722619547228 time 722657360637 state 0x2 ref 0 00000100:00100000:2.0:1789760738.362569:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760738.362657:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495803136 00000100:00100000:1.0:1789760738.362666:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495803136 00000100:00100000:1.0:1789760738.362713:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722619547228 time 722657360637 state 0x0 ref 1 00000100:00100000:1.0:1789760738.362728:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722657621053 time 722657360637 state 0x1 ref 1 00000100:00100000:1.0:1789760738.362785:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722657621053 time 722683500746 state 0x0 ref 1 00000100:00100000:1.0:1789760738.362797:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04292a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495803136:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760738.362844:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23142400, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:1.0:1789760738.362852:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x493674835454624a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.362868:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760738.362877:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.362884:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x493674835454624a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.362898:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x493674835454624a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760738.392210:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972967 is committed 00002000:00010000:2.0:1789760738.392448:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23142400 -> 23146496 00010000:00010000:2.0:1789760738.392465:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x493674835454624a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.392479:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x493674835454624a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.392487:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x493674835454624a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.392510:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760738.392515:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x493674835454624a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.392525:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x493674835454624a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.392548:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x493674835454624a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.392556:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x493674835454624a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.392649:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04292a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495803136:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29850us (30084us total) trans 4294972967 rc 0/0 00000100:00100000:2.0:1789760738.392660:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5679 00000100:00100000:2.0:1789760738.392671:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722657621053 time 722683500746 state 0x2 ref 0 00000100:00100000:1.0:1789760738.398189:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760738.398515:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495803264 00000100:00100000:3.0:1789760738.398525:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495803264 00000100:00100000:3.0:1789760738.398580:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722657621053 time 722683500746 state 0x0 ref 1 00000100:00100000:3.0:1789760738.398600:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722683761162 time 722683500746 state 0x1 ref 1 00000100:00100000:3.0:1789760738.398627:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722683761162 time 722719343673 state 0x0 ref 1 00000100:00100000:3.0:1789760738.398639:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495803264:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760738.398683:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23146496, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:3.0:1789760738.398692:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546251 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.398708:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760738.398712:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.398718:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546251 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.398732:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546251 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760738.429695:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972968 is committed 00002000:00010000:2.0:1789760738.429846:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23146496 -> 23150592 00010000:00010000:2.0:1789760738.429861:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546251 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.429874:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546251 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.429889:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546251 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.429896:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760738.429901:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546251 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.429909:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546251 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.429926:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546251 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.429933:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546251 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.430014:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495803264:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31372us (31828us total) trans 4294972968 rc 0/0 00000100:00100000:2.0:1789760738.430024:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5680 00000100:00100000:2.0:1789760738.430033:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722683761162 time 722719343673 state 0x2 ref 0 00000100:00100000:0.0:1789760738.434815:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760738.434927:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495803392 00000100:00100000:2.0:1789760738.434937:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495803392 00000100:00100000:2.0:1789760738.434989:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722683761162 time 722719343673 state 0x0 ref 1 00000100:00100000:2.0:1789760738.435003:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722719604089 time 722719343673 state 0x1 ref 1 00000100:00100000:2.0:1789760738.435034:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722719604089 time 722755750888 state 0x0 ref 1 00000100:00100000:2.0:1789760738.435179:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f923100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495803392:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760738.435238:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23150592, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:2.0:1789760738.435247:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546258 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.435263:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760738.435267:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.435273:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546258 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.435349:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546258 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760738.476240:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23150592 -> 23154688 00010000:00010000:2.0:1789760738.476259:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546258 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.476271:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546258 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.476299:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546258 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.476307:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760738.476313:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546258 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.476321:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546258 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.476340:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546258 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.476348:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546258 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.476424:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f923100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495803392:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41375us (41614us total) trans 4294972969 rc 0/0 00000100:00100000:2.0:1789760738.476435:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5681 00000100:00100000:2.0:1789760738.476444:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722719604089 time 722755750888 state 0x2 ref 0 00000001:00080000:0.0:1789760738.476766:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972969 is committed 00000100:00100000:0.0:1789760738.480462:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760738.480983:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495803520 00000100:00100000:2.0:1789760738.480993:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495803520 00000100:00100000:2.0:1789760738.481043:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722719604089 time 722755750888 state 0x0 ref 1 00000100:00100000:2.0:1789760738.481059:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722756011304 time 722755750888 state 0x1 ref 1 00000100:00100000:0.0:1789760738.481239:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722756011304 time 722801953517 state 0x0 ref 1 00000100:00100000:0.0:1789760738.481253:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f922300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495803520:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760738.481322:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23154688, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:0.0:1789760738.481331:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454625f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.481347:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760738.481352:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.481359:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454625f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.481372:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454625f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760738.514079:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972970 is committed 00002000:00010000:0.0:1789760738.514422:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23154688 -> 23158784 00010000:00010000:0.0:1789760738.514437:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454625f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.514450:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454625f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.514459:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454625f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.514467:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760738.514472:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454625f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.514481:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454625f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.514567:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454625f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.514576:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454625f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.514885:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f922300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495803520:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33630us (34435us total) trans 4294972970 rc 0/0 00000100:00100000:0.0:1789760738.514897:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5682 00000100:00100000:0.0:1789760738.514907:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722756011304 time 722801953517 state 0x2 ref 0 00000100:00100000:1.0:1789760738.522274:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760738.522583:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495803648 00000100:00100000:2.0:1789760738.522601:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495803648 00000100:00100000:2.0:1789760738.522656:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722756011304 time 722801953517 state 0x0 ref 1 00000100:00100000:2.0:1789760738.522675:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722802213933 time 722801953517 state 0x1 ref 1 00000100:00100000:2.0:1789760738.522705:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722802213933 time 722843421735 state 0x0 ref 1 00000100:00100000:2.0:1789760738.522718:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495803648:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760738.522765:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23158784, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:2.0:1789760738.522774:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546266 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.522791:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760738.522795:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.522800:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546266 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.522813:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546266 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760738.568477:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972971 is committed 00002000:00010000:0.0:1789760738.570350:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23158784 -> 23162880 00010000:00010000:0.0:1789760738.570369:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546266 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.570383:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546266 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.570391:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546266 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.570398:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760738.570402:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546266 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.570410:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546266 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.570428:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546266 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.570558:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546266 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.570730:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495803648:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48008us (48459us total) trans 4294972971 rc 0/0 00000100:00100000:0.0:1789760738.570741:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5683 00000100:00100000:0.0:1789760738.570752:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722802213933 time 722843421735 state 0x2 ref 0 00000100:00100000:2.0:1789760738.579136:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760738.581795:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495803776 00000100:00100000:0.0:1789760738.581810:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495803776 00000100:00100000:0.0:1789760738.581883:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722802213933 time 722843421735 state 0x0 ref 1 00000100:00100000:0.0:1789760738.581899:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722843682151 time 722843421735 state 0x1 ref 1 00000100:00100000:0.0:1789760738.581935:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722843682151 time 722902651585 state 0x0 ref 1 00000100:00100000:0.0:1789760738.581949:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495803776:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760738.582000:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23162880, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:0.0:1789760738.582010:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454626d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.582027:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760738.582032:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.582039:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454626d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.582337:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454626d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760738.614339:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972972 is committed 00002000:00010000:1.0:1789760738.614542:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23162880 -> 23166976 00010000:00010000:1.0:1789760738.614556:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454626d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.614569:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454626d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.614578:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454626d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.614590:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760738.614595:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454626d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.614603:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454626d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.614623:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454626d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.614631:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454626d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760738.614752:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495803776:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32802us (35622us total) trans 4294972972 rc 0/0 00000100:00100000:1.0:1789760738.614763:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5684 00000100:00100000:1.0:1789760738.614772:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722843682151 time 722902651585 state 0x2 ref 0 00000100:00100000:2.0:1789760738.620715:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760738.620811:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495803904 00000100:00100000:0.0:1789760738.620830:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495803904 00000100:00100000:0.0:1789760738.620887:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722843682151 time 722902651585 state 0x0 ref 1 00000100:00100000:0.0:1789760738.620903:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722902912001 time 722902651585 state 0x1 ref 1 00000100:00100000:0.0:1789760738.620971:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722902912001 time 722941687221 state 0x0 ref 1 00000100:00100000:0.0:1789760738.620984:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495803904:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760738.621032:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23166976, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:0.0:1789760738.621041:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546274 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.621189:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760738.621195:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.621201:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546274 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.621214:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546274 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760738.650393:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972973 is committed 00002000:00010000:2.0:1789760738.650565:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23166976 -> 23171072 00002000:00010000:2.0:1789760738.650570:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760737 -> 1789760738 00002000:00010000:2.0:1789760738.650573:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760737 -> 1789760738 00010000:00010000:2.0:1789760738.650585:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546274 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.650605:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546274 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.650614:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546274 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.650622:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94aa00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760738.650627:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546274 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.650637:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546274 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.650658:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546274 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.650666:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546274 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.650757:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495803904:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29770us (30046us total) trans 4294972973 rc 0/0 00000100:00100000:2.0:1789760738.650767:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5685 00000100:00100000:2.0:1789760738.650777:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722902912001 time 722941687221 state 0x2 ref 0 00000100:00100000:2.0:1789760738.655698:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760738.655782:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495804032 00000100:00100000:0.0:1789760738.655792:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495804032 00000100:00100000:0.0:1789760738.655847:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722902912001 time 722941687221 state 0x0 ref 1 00000100:00100000:0.0:1789760738.655862:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722941947637 time 722941687221 state 0x1 ref 1 00000100:00100000:0.0:1789760738.655891:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722941947637 time 722976607880 state 0x0 ref 1 00000100:00100000:0.0:1789760738.655904:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495804032:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760738.655952:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23171072, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:0.0:1789760738.655961:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454627b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.655976:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760738.655981:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.655987:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454627b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.656001:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454627b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760738.701129:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972974 is committed 00002000:00010000:2.0:1789760738.702632:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23171072 -> 23175168 00010000:00010000:2.0:1789760738.702651:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454627b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.702663:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454627b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.702671:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454627b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.702677:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760738.702681:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454627b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.702689:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454627b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.702710:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454627b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.702717:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454627b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.702925:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495804032:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47017us (47229us total) trans 4294972974 rc 0/0 00000100:00100000:2.0:1789760738.702938:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5686 00000100:00100000:2.0:1789760738.702948:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722941947637 time 722976607880 state 0x2 ref 0 00000100:00100000:2.0:1789760738.711650:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760738.711848:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495804160 00000100:00100000:3.0:1789760738.711858:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495804160 00000100:00100000:3.0:1789760738.711912:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722941947637 time 722976607880 state 0x0 ref 1 00000100:00100000:3.0:1789760738.711929:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722976868296 time 722976607880 state 0x1 ref 1 00000100:00100000:3.0:1789760738.711961:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722976868296 time 723032677681 state 0x0 ref 1 00000100:00100000:3.0:1789760738.711974:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495804160:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760738.712020:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23175168, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:3.0:1789760738.712029:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546282 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.712044:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760738.712048:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.712070:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546282 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.712092:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546282 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760738.760957:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23175168 -> 23179264 00010000:00010000:1.0:1789760738.760988:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546282 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.761002:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546282 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.761017:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546282 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.761026:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760738.761035:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546282 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.761053:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546282 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.761081:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546282 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.761088:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546282 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760738.761168:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972975 is committed 00000100:00100000:1.0:1789760738.761276:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495804160:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49300us (49631us total) trans 4294972975 rc 0/0 00000100:00100000:1.0:1789760738.761321:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5687 00000100:00100000:1.0:1789760738.761332:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722976868296 time 723032677681 state 0x2 ref 0 00000100:00100000:2.0:1789760738.764273:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760738.764466:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495804288 00000100:00100000:0.0:1789760738.764476:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495804288 00000100:00100000:0.0:1789760738.764600:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 722976868296 time 723032677681 state 0x0 ref 1 00000100:00100000:0.0:1789760738.764616:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723032938097 time 723032677681 state 0x1 ref 1 00000100:00100000:0.0:1789760738.764651:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723032938097 time 723085367443 state 0x0 ref 1 00000100:00100000:0.0:1789760738.764664:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495804288:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760738.764714:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23179264, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:0.0:1789760738.764722:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546289 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.764736:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760738.764741:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.764747:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546289 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.764760:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546289 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.804944:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760738.805000:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760738.805034:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760738.805087:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495804544 00000100:00100000:2.0:1789760738.805098:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495804544 00000100:00100000:3.0:1789760738.805297:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495804416 00000100:00100000:3.0:1789760738.805301:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495804416 00000100:00100000:3.0:1789760738.805336:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 400 00000100:00100000:3.0:1789760738.805345:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929012312450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2356:x1876699495804416:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:3.0:1789760738.805428:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929012312450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2356:x1876699495804416:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 83us (426us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760738.805437:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 400 00000100:00100000:2.0:1789760738.805757:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495804672 00000100:00100000:2.0:1789760738.805762:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495804672 00000100:00100000:2.0:1789760738.805796:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 480 00000100:00100000:2.0:1789760738.805803:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:17022965-1e4f-46fb-915c-61730096b8e2+8:2356:x1876699495804544:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:2.0:1789760738.805841:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:17022965-1e4f-46fb-915c-61730096b8e2+8:2356:x1876699495804544:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 39us (908us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760738.805847:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 480 00000100:00100000:2.0:1789760738.805860:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 481 00000100:00100000:2.0:1789760738.805866:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2356:x1876699495804672:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:2.0:1789760738.805888:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2356:x1876699495804672:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 23us (853us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760738.805893:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 481 00000100:00080000:2.0:1789760738.805909:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:2.0:1789760738.816948:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972976 is committed 00002000:00010000:0.0:1789760738.817101:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23179264 -> 23183360 00010000:00010000:0.0:1789760738.817117:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546289 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.817137:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546289 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.817147:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546289 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.817268:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ae00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760738.817274:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546289 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.817315:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546289 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.817334:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546289 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.817340:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546289 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.817620:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495804288:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52954us (53350us total) trans 4294972976 rc 0/0 00000100:00100000:0.0:1789760738.817633:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5688 00000100:00100000:0.0:1789760738.817707:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723032938097 time 723085367443 state 0x2 ref 0 00000100:00100000:3.0:1789760738.821977:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760738.822085:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495804800 00000100:00100000:2.0:1789760738.822098:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495804800 00000100:00100000:2.0:1789760738.822166:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723032938097 time 723085367443 state 0x0 ref 1 00000100:00100000:2.0:1789760738.822182:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723085627859 time 723085367443 state 0x1 ref 1 00000100:00100000:2.0:1789760738.822215:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723085627859 time 723142932275 state 0x0 ref 1 00000100:00100000:2.0:1789760738.822227:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495804800:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760738.822273:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23183360, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:2.0:1789760738.822311:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546290 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.822329:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760738.822333:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.822342:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546290 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.822358:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546290 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760738.870550:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23183360 -> 23187456 00010000:00010000:2.0:1789760738.870571:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546290 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.870585:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546290 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.870593:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546290 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.870599:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760738.870605:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546290 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.870613:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546290 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.870632:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546290 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.870640:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546290 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.870747:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495804800:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48515us (48773us total) trans 4294972977 rc 0/0 00000100:00100000:2.0:1789760738.870758:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5689 00000100:00100000:2.0:1789760738.870768:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723085627859 time 723142932275 state 0x2 ref 0 00000001:00080000:0.0:1789760738.870923:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972977 is committed 00000100:00100000:2.0:1789760738.875367:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760738.875520:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495804928 00000100:00100000:1.0:1789760738.875602:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495804928 00000100:00100000:1.0:1789760738.875659:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723085627859 time 723142932275 state 0x0 ref 1 00000100:00100000:1.0:1789760738.875674:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723143192691 time 723142932275 state 0x1 ref 1 00000100:00100000:1.0:1789760738.875737:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723143192691 time 723196453146 state 0x0 ref 1 00000100:00100000:1.0:1789760738.875751:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495804928:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760738.875801:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23187456, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:1.0:1789760738.875811:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546297 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.875826:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760738.875830:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.875837:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546297 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.875851:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546297 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760738.940560:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972978 is committed 00002000:00010000:0.0:1789760738.940851:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23187456 -> 23191552 00010000:00010000:0.0:1789760738.940868:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546297 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.940883:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546297 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.941024:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546297 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.941038:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760738.941043:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546297 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.941052:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546297 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.941276:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546297 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.941434:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546297 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.941528:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495804928:12345-192.168.203.9@tcp:4:dd.0 Request processed in 65775us (66168us total) trans 4294972978 rc 0/0 00000100:00100000:0.0:1789760738.941539:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5690 00000100:00100000:0.0:1789760738.941550:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723143192691 time 723196453146 state 0x2 ref 0 00000100:00100000:1.0:1789760738.945475:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760738.945573:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495805056 00000100:00100000:0.0:1789760738.945584:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495805056 00000100:00100000:0.0:1789760738.945890:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723143192691 time 723196453146 state 0x0 ref 1 00000100:00100000:0.0:1789760738.945907:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723196713562 time 723196453146 state 0x1 ref 1 00000100:00100000:0.0:1789760738.945942:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723196713562 time 723266658805 state 0x0 ref 1 00000100:00100000:0.0:1789760738.945960:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495805056:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760738.946012:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23191552, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:0.0:1789760738.946021:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454629e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.946037:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760738.946041:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.946048:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454629e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.946072:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454629e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760738.994208:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972979 is committed 00002000:00010000:2.0:1789760738.994340:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23191552 -> 23195648 00010000:00010000:2.0:1789760738.994355:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454629e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.994367:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454629e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.994376:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454629e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.994390:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ae00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760738.994395:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454629e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.994404:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454629e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.994422:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454629e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.994429:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454629e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.994517:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495805056:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48557us (49048us total) trans 4294972979 rc 0/0 00000100:00100000:2.0:1789760738.994526:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5691 00000100:00100000:2.0:1789760738.994536:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723196713562 time 723266658805 state 0x2 ref 0 00000100:00100000:3.0:1789760739.003134:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760739.003229:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495805184 00000100:00100000:2.0:1789760739.003237:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495805184 00000100:00100000:2.0:1789760739.003329:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723196713562 time 723266658805 state 0x0 ref 1 00000100:00100000:2.0:1789760739.003343:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723266919221 time 723266658805 state 0x1 ref 1 00000100:00080000:1.0:1789760739.003510:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760739.003599:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723266919221 time 723324313165 state 0x0 ref 1 00000100:00100000:0.0:1789760739.003615:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495805184:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760739.003671:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23195648, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:0.0:1789760739.003681:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545462a5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.003697:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760739.003702:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760739.003709:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545462a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.003728:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545462a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760739.040467:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972980 is committed 00002000:00010000:0.0:1789760739.040583:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23195648 -> 23199744 00010000:00010000:0.0:1789760739.040598:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545462a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.040611:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545462a5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.040620:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545462a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.040627:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94aa00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760739.040632:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545462a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.040641:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545462a5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.040661:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545462a5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.040669:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545462a5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.040785:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495805184:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37168us (37653us total) trans 4294972980 rc 0/0 00000100:00100000:0.0:1789760739.040797:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5692 00000100:00100000:0.0:1789760739.040806:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723266919221 time 723324313165 state 0x2 ref 0 00000100:00100000:0.0:1789760739.044623:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760739.044719:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495805312 00000100:00100000:1.0:1789760739.044729:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495805312 00000100:00100000:1.0:1789760739.044787:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723266919221 time 723324313165 state 0x0 ref 1 00000100:00100000:1.0:1789760739.044802:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723324573581 time 723324313165 state 0x1 ref 1 00000100:00100000:1.0:1789760739.044833:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723324573581 time 723365549072 state 0x0 ref 1 00000100:00100000:1.0:1789760739.044847:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495805312:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760739.044896:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23199744, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:1.0:1789760739.044907:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545462ac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.044923:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760739.044929:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.044935:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545462ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.044949:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545462ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760739.101893:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972981 is committed 00002000:00010000:0.0:1789760739.102045:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23199744 -> 23203840 00010000:00010000:0.0:1789760739.102070:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545462ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.102084:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545462ac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.102093:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545462ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.102101:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760739.102106:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545462ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.102115:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545462ac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.102136:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545462ac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.102147:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545462ac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.102240:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495805312:12345-192.168.203.9@tcp:4:dd.0 Request processed in 57390us (57622us total) trans 4294972981 rc 0/0 00000100:00100000:0.0:1789760739.102249:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5693 00000100:00100000:0.0:1789760739.102259:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723324573581 time 723365549072 state 0x2 ref 0 00000100:00100000:1.0:1789760739.111538:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760739.111618:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495805440 00000100:00100000:2.0:1789760739.111627:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495805440 00000100:00100000:2.0:1789760739.111690:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723324573581 time 723365549072 state 0x0 ref 1 00000100:00100000:2.0:1789760739.111705:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723365809488 time 723365549072 state 0x1 ref 1 00000100:00100000:2.0:1789760739.111735:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723365809488 time 723432452234 state 0x0 ref 1 00000100:00100000:2.0:1789760739.111748:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495805440:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760739.111798:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23203840, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:2.0:1789760739.111807:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545462b3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.111823:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760739.111827:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.111834:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545462b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.111846:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545462b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760739.164043:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972982 is committed 00002000:00010000:2.0:1789760739.166147:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23203840 -> 23207936 00010000:00010000:2.0:1789760739.166164:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545462b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.166176:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545462b3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.166401:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545462b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.166412:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760739.166418:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545462b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.166428:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545462b3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.166457:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545462b3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.166465:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545462b3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.166498:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789760739.166558:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495805440:12345-192.168.203.9@tcp:4:dd.0 Request processed in 54806us (55023us total) trans 4294972982 rc 0/0 00000100:00100000:2.0:1789760739.166568:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5694 00000100:00100000:2.0:1789760739.166578:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723365809488 time 723432452234 state 0x2 ref 0 00000100:00100000:1.0:1789760739.171364:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760739.171447:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495805568 00000100:00100000:0.0:1789760739.171456:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495805568 00000100:00100000:0.0:1789760739.171512:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723365809488 time 723432452234 state 0x0 ref 1 00000100:00100000:0.0:1789760739.171527:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723432712650 time 723432452234 state 0x1 ref 1 00000100:00100000:0.0:1789760739.171560:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723432712650 time 723492277124 state 0x0 ref 1 00000100:00100000:0.0:1789760739.171574:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495805568:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760739.171628:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23207936, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:0.0:1789760739.171637:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545462ba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.171668:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760739.171673:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760739.171679:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545462ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.171698:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545462ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760739.222805:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972983 is committed 00002000:00010000:1.0:1789760739.223176:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23207936 -> 23212032 00010000:00010000:1.0:1789760739.223193:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545462ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.223207:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545462ba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.223217:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545462ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.223226:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760739.223231:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545462ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.223241:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545462ba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.223263:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545462ba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.223272:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545462ba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.223467:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495805568:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51890us (52108us total) trans 4294972983 rc 0/0 00000100:00100000:1.0:1789760739.223478:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5695 00000100:00100000:1.0:1789760739.223488:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723432712650 time 723492277124 state 0x2 ref 0 00000100:00100000:0.0:1789760739.229095:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760739.229193:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495805696 00000100:00100000:1.0:1789760739.229203:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495805696 00000100:00100000:1.0:1789760739.229256:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723432712650 time 723492277124 state 0x0 ref 1 00000100:00100000:1.0:1789760739.229272:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723492537540 time 723492277124 state 0x1 ref 1 00000100:00100000:1.0:1789760739.229326:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723492537540 time 723550042208 state 0x0 ref 1 00000100:00100000:2.0:1789760739.229382:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495805696:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760739.229433:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23212032, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:2.0:1789760739.229442:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545462c1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.229458:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760739.229463:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.229470:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545462c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.229486:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545462c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760739.295377:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23212032 -> 23216128 00010000:00010000:2.0:1789760739.295408:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545462c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.295421:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545462c1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.295429:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545462c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.295436:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760739.295442:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545462c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.295451:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545462c1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.295471:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545462c1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.295485:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545462c1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.295583:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495805696:12345-192.168.203.9@tcp:4:dd.0 Request processed in 66237us (66492us total) trans 4294972984 rc 0/0 00000100:00100000:2.0:1789760739.295594:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5696 00000100:00100000:2.0:1789760739.295604:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723492537540 time 723550042208 state 0x2 ref 0 00000001:00080000:2.0:1789760739.295661:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972984 is committed 00000100:00100000:2.0:1789760739.310439:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760739.310535:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495805824 00000100:00100000:0.0:1789760739.310546:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495805824 00000100:00100000:0.0:1789760739.310603:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723492537540 time 723550042208 state 0x0 ref 1 00000100:00100000:0.0:1789760739.310618:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723550302624 time 723550042208 state 0x1 ref 1 00000100:00100000:0.0:1789760739.310646:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723550302624 time 723631362433 state 0x0 ref 1 00000100:00100000:0.0:1789760739.310660:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495805824:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760739.310708:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23216128, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:0.0:1789760739.310719:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545462c8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.310735:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760739.310740:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760739.310747:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545462c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.310761:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545462c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760739.347022:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972985 is committed 00002000:00010000:3.0:1789760739.347254:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23216128 -> 23220224 00010000:00010000:3.0:1789760739.347270:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545462c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.347351:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545462c8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.347360:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545462c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.347368:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760739.347373:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545462c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.347388:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545462c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.347559:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545462c8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.347569:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545462c8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.347699:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495805824:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37037us (37265us total) trans 4294972985 rc 0/0 00000100:00100000:3.0:1789760739.347710:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5697 00000100:00100000:3.0:1789760739.347719:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723550302624 time 723631362433 state 0x2 ref 0 00000100:00100000:3.0:1789760739.357086:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760739.357203:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495805952 00000100:00100000:1.0:1789760739.357213:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495805952 00000100:00100000:1.0:1789760739.357271:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723550302624 time 723631362433 state 0x0 ref 1 00000100:00100000:1.0:1789760739.357322:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723631622849 time 723631362433 state 0x1 ref 1 00000100:00100000:1.0:1789760739.357348:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723631622849 time 723678064832 state 0x0 ref 1 00000100:00100000:1.0:1789760739.357361:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370bd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495805952:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760739.357410:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23220224, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:1.0:1789760739.357420:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545462cf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.357436:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760739.357440:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.357447:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545462cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.357461:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545462cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760739.419143:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972986 is committed 00002000:00010000:3.0:1789760739.419338:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23220224 -> 23224320 00010000:00010000:3.0:1789760739.419354:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545462cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.419368:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545462cf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.419378:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545462cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.419390:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760739.419395:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545462cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.419403:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545462cf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.419422:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545462cf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.419431:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545462cf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.419746:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370bd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495805952:12345-192.168.203.9@tcp:4:dd.0 Request processed in 62382us (62664us total) trans 4294972986 rc 0/0 00000100:00100000:3.0:1789760739.419758:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5698 00000100:00100000:3.0:1789760739.419767:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723631622849 time 723678064832 state 0x2 ref 0 00000100:00100000:0.0:1789760739.423226:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760739.423885:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495806080 00000100:00100000:2.0:1789760739.423896:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495806080 00000100:00100000:2.0:1789760739.423954:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723631622849 time 723678064832 state 0x0 ref 1 00000100:00100000:2.0:1789760739.423968:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723678325248 time 723678064832 state 0x1 ref 1 00000100:00100000:2.0:1789760739.424005:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723678325248 time 723744716148 state 0x0 ref 1 00000100:00100000:2.0:1789760739.424017:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495806080:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760739.424074:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23224320, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:2.0:1789760739.424083:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545462d6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.424101:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760739.424105:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.424113:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545462d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.424126:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545462d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760739.458358:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972987 is committed 00002000:00010000:0.0:1789760739.458466:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23224320 -> 23228416 00010000:00010000:0.0:1789760739.458483:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545462d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.458497:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545462d6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.458507:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545462d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.458519:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760739.458524:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545462d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.458533:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545462d6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.458564:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545462d6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.458573:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545462d6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.458677:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495806080:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34657us (35455us total) trans 4294972987 rc 0/0 00000100:00100000:0.0:1789760739.458687:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5699 00000100:00100000:0.0:1789760739.458697:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723678325248 time 723744716148 state 0x2 ref 0 00000100:00100000:2.0:1789760739.469157:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760739.469258:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495806208 00000100:00100000:3.0:1789760739.469267:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495806208 00000100:00100000:3.0:1789760739.469360:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723678325248 time 723744716148 state 0x0 ref 1 00000100:00100000:3.0:1789760739.469375:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723744976564 time 723744716148 state 0x1 ref 1 00000100:00100000:3.0:1789760739.469412:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723744976564 time 723790129328 state 0x0 ref 1 00000100:00100000:3.0:1789760739.469428:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495806208:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760739.469475:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23228416, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:3.0:1789760739.469485:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545462dd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.469502:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760739.469506:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.469513:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545462dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.469527:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545462dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760739.500636:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972988 is committed 00002000:00010000:0.0:1789760739.500771:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23228416 -> 23232512 00010000:00010000:0.0:1789760739.500787:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545462dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.500800:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545462dd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.500809:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545462dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.500816:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760739.500820:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545462dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.500829:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545462dd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.500851:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545462dd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.500863:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545462dd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.500992:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495806208:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31561us (31839us total) trans 4294972988 rc 0/0 00000100:00100000:0.0:1789760739.501003:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5700 00000100:00100000:0.0:1789760739.501013:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723744976564 time 723790129328 state 0x2 ref 0 00000100:00100000:1.0:1789760739.508601:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760739.508719:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495806336 00000100:00100000:3.0:1789760739.508729:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495806336 00000100:00100000:3.0:1789760739.508783:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723744976564 time 723790129328 state 0x0 ref 1 00000100:00100000:3.0:1789760739.508799:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723790389744 time 723790129328 state 0x1 ref 1 00000100:00100000:3.0:1789760739.508832:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723790389744 time 723829549058 state 0x0 ref 1 00000100:00100000:3.0:1789760739.508845:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495806336:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760739.508895:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23232512, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:3.0:1789760739.508904:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545462e4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.508925:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760739.508934:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.508941:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545462e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.508955:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545462e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760739.534818:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972989 is committed 00002000:00010000:3.0:1789760739.534895:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23232512 -> 23236608 00010000:00010000:3.0:1789760739.534910:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545462e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.534923:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545462e4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.534932:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545462e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.534939:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760739.534944:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545462e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.534954:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545462e4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.534973:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545462e4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.534981:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545462e4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.535176:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495806336:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26328us (26578us total) trans 4294972989 rc 0/0 00000100:00100000:3.0:1789760739.535186:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5701 00000100:00100000:3.0:1789760739.535197:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723790389744 time 723829549058 state 0x2 ref 0 00000100:00100000:2.0:1789760739.539653:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760739.539908:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495806464 00000100:00100000:0.0:1789760739.539920:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495806464 00000100:00100000:0.0:1789760739.539974:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723790389744 time 723829549058 state 0x0 ref 1 00000100:00100000:0.0:1789760739.539989:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723829809474 time 723829549058 state 0x1 ref 1 00000100:00100000:0.0:1789760739.540020:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723829809474 time 723860736233 state 0x0 ref 1 00000100:00100000:0.0:1789760739.540032:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495806464:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760739.540099:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23236608, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:0.0:1789760739.540109:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545462eb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.540125:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760739.540129:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760739.540135:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545462eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.540155:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545462eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760739.568182:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23236608 -> 23240704 00010000:00010000:0.0:1789760739.568200:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545462eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.568214:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545462eb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.568223:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545462eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.568231:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760739.568235:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545462eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.568244:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545462eb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.568264:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545462eb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.568271:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545462eb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760739.568511:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972990 is committed 00000100:00100000:0.0:1789760739.568565:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495806464:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28530us (28915us total) trans 4294972990 rc 0/0 00000100:00100000:0.0:1789760739.568577:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5702 00000100:00100000:0.0:1789760739.568587:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723829809474 time 723860736233 state 0x2 ref 0 00000100:00100000:2.0:1789760739.572257:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760739.572435:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495806592 00000100:00100000:0.0:1789760739.572445:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495806592 00000100:00100000:0.0:1789760739.572499:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723829809474 time 723860736233 state 0x0 ref 1 00000100:00100000:0.0:1789760739.572514:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723860996649 time 723860736233 state 0x1 ref 1 00000100:00100000:0.0:1789760739.572581:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723860996649 time 723893296654 state 0x0 ref 1 00000100:00100000:0.0:1789760739.572594:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495806592:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760739.572643:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23240704, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:0.0:1789760739.572653:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545462f2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.572669:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760739.572673:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760739.572680:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545462f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.572694:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545462f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760739.607169:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972991 is committed 00002000:00010000:0.0:1789760739.607258:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23240704 -> 23244800 00010000:00010000:0.0:1789760739.607274:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545462f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.607366:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545462f2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.607374:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545462f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.607455:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760739.607461:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545462f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.607471:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545462f2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.607493:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545462f2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.607500:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545462f2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.607688:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495806592:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35089us (35434us total) trans 4294972991 rc 0/0 00000100:00100000:2.0:1789760739.607700:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5703 00000100:00100000:2.0:1789760739.607709:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723860996649 time 723893296654 state 0x2 ref 0 00000100:00100000:1.0:1789760739.611708:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760739.611787:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495806720 00000100:00100000:3.0:1789760739.611797:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495806720 00000100:00100000:3.0:1789760739.611848:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723860996649 time 723893296654 state 0x0 ref 1 00000100:00100000:3.0:1789760739.611864:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723893557070 time 723893296654 state 0x1 ref 1 00000100:00100000:3.0:1789760739.611895:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723893557070 time 723932612080 state 0x0 ref 1 00000100:00100000:3.0:1789760739.611907:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495806720:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760739.611953:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23244800, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:3.0:1789760739.611962:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545462f9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.611979:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760739.611983:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.611990:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545462f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.612010:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545462f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760739.647521:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23244800 -> 23248896 00010000:00010000:2.0:1789760739.647542:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545462f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.647554:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545462f9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.647562:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545462f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.647568:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760739.647574:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545462f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.647583:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545462f9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.647602:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545462f9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.647610:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545462f9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.647783:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495806720:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35873us (36080us total) trans 4294972992 rc 0/0 00000100:00100000:2.0:1789760739.647796:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5704 00000100:00100000:2.0:1789760739.647806:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723893557070 time 723932612080 state 0x2 ref 0 00000001:00080000:0.0:1789760739.647891:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972992 is committed 00000100:00100000:0.0:1789760739.654776:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760739.654874:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495806848 00000100:00100000:2.0:1789760739.654884:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495806848 00000100:00100000:2.0:1789760739.654938:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723893557070 time 723932612080 state 0x0 ref 1 00000100:00100000:2.0:1789760739.654954:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723932872496 time 723932612080 state 0x1 ref 1 00000100:00100000:2.0:1789760739.654985:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723932872496 time 723975701651 state 0x0 ref 1 00000100:00100000:2.0:1789760739.654998:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495806848:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760739.655066:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23248896, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:2.0:1789760739.655075:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546300 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.655090:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760739.655094:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.655101:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546300 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.655114:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546300 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760739.684714:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972993 is committed 00002000:00010000:0.0:1789760739.685107:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23248896 -> 23252992 00002000:00010000:0.0:1789760739.685112:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760738 -> 1789760739 00002000:00010000:0.0:1789760739.685115:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760738 -> 1789760739 00010000:00010000:0.0:1789760739.685128:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546300 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.685138:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546300 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.685144:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546300 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.685148:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760739.685152:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546300 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.685158:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546300 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.685175:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546300 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.685179:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546300 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.685358:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495806848:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30357us (30587us total) trans 4294972993 rc 0/0 00000100:00100000:0.0:1789760739.685373:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5705 00000100:00100000:0.0:1789760739.685382:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723932872496 time 723975701651 state 0x2 ref 0 00000100:00100000:1.0:1789760739.688717:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760739.689261:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495806976 00000100:00100000:2.0:1789760739.689272:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495806976 00000100:00100000:2.0:1789760739.690217:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723932872496 time 723975701651 state 0x0 ref 1 00000100:00100000:2.0:1789760739.690235:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723975962067 time 723975701651 state 0x1 ref 1 00000100:00100000:2.0:1789760739.690272:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723975962067 time 724010988335 state 0x0 ref 1 00000100:00100000:2.0:1789760739.690304:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495806976:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760739.690353:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23252992, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:2.0:1789760739.690361:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546307 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.690378:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760739.690382:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.690388:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546307 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.690403:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546307 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760739.781003:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972994 is committed 00002000:00010000:2.0:1789760739.781349:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23252992 -> 23257088 00010000:00010000:2.0:1789760739.781367:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546307 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.781379:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546307 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.781389:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546307 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.781396:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760739.781402:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546307 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.781411:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546307 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.781431:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546307 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.781440:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546307 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.781526:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495806976:12345-192.168.203.9@tcp:4:dd.0 Request processed in 91222us (92815us total) trans 4294972994 rc 0/0 00000100:00100000:2.0:1789760739.781536:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5706 00000100:00100000:2.0:1789760739.781545:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723975962067 time 724010988335 state 0x2 ref 0 00000100:00100000:1.0:1789760739.792441:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760739.794624:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495807104 00000100:00100000:2.0:1789760739.794638:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495807104 00000100:00100000:3.0:1789760739.794984:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 723975962067 time 724010988335 state 0x0 ref 1 00000100:00100000:3.0:1789760739.795000:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724011248751 time 724010988335 state 0x1 ref 1 00000100:00100000:3.0:1789760739.795030:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724011248751 time 724115747435 state 0x0 ref 1 00000100:00100000:3.0:1789760739.795050:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495807104:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760739.795103:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23257088, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:3.0:1789760739.795113:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454630e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.795129:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760739.795133:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.795139:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454630e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.795153:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454630e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760739.873026:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972995 is committed 00002000:00010000:0.0:1789760739.878341:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23257088 -> 23261184 00010000:00010000:0.0:1789760739.878362:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454630e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.878375:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454630e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.878386:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454630e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.878394:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760739.878399:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454630e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.878408:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454630e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.878429:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454630e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.878437:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454630e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.878596:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495807104:12345-192.168.203.9@tcp:4:dd.0 Request processed in 83543us (86160us total) trans 4294972995 rc 0/0 00000100:00100000:0.0:1789760739.878607:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5707 00000100:00100000:0.0:1789760739.878616:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724011248751 time 724115747435 state 0x2 ref 0 00000100:00100000:0.0:1789760739.901487:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760739.901592:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495807232 00000100:00100000:2.0:1789760739.901602:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495807232 00000100:00100000:2.0:1789760739.901655:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724011248751 time 724115747435 state 0x0 ref 1 00000100:00100000:2.0:1789760739.901670:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724116007851 time 724115747435 state 0x1 ref 1 00000100:00100000:2.0:1789760739.901701:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724116007851 time 724222417691 state 0x0 ref 1 00000100:00100000:2.0:1789760739.901714:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03095500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495807232:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760739.901763:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23261184, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:2.0:1789760739.901772:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546315 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.901788:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760739.901792:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.901799:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546315 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.901817:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546315 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760739.936695:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23261184 -> 23265280 00010000:00010000:2.0:1789760739.936714:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546315 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.936727:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546315 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.936737:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546315 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.936746:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760739.936751:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546315 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.936760:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546315 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.936789:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546315 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.936797:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546315 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.936899:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495807232:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35183us (35416us total) trans 4294972996 rc 0/0 00000100:00100000:2.0:1789760739.936909:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5708 00000100:00100000:2.0:1789760739.936919:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724116007851 time 724222417691 state 0x2 ref 0 00000001:00080000:1.0:1789760739.937500:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972996 is committed 00000100:00100000:1.0:1789760739.940693:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760739.941043:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495807360 00000100:00100000:2.0:1789760739.941061:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495807360 00000100:00100000:2.0:1789760739.941115:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724116007851 time 724222417691 state 0x0 ref 1 00000100:00100000:2.0:1789760739.941129:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724222678107 time 724222417691 state 0x1 ref 1 00000100:00100000:2.0:1789760739.941163:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724222678107 time 724261879028 state 0x0 ref 1 00000100:00100000:2.0:1789760739.941182:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495807360:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760739.941230:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23265280, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:2.0:1789760739.941239:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454631c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.941256:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760739.941260:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.941267:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454631c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.941319:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454631c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760739.973635:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972997 is committed 00002000:00010000:3.0:1789760739.973702:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23265280 -> 23269376 00010000:00010000:3.0:1789760739.973719:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454631c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.973737:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454631c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.973745:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454631c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.973753:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760739.973758:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454631c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.973767:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454631c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.973847:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454631c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.973855:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454631c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.973930:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495807360:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32754us (33243us total) trans 4294972997 rc 0/0 00000100:00100000:3.0:1789760739.973937:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5709 00000100:00100000:3.0:1789760739.973944:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724222678107 time 724261879028 state 0x2 ref 0 00000100:00100000:0.0:1789760739.983038:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760739.983160:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495807488 00000100:00100000:3.0:1789760739.983171:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495807488 00000100:00100000:3.0:1789760739.983227:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724222678107 time 724261879028 state 0x0 ref 1 00000100:00100000:3.0:1789760739.983242:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724262139444 time 724261879028 state 0x1 ref 1 00000100:00100000:3.0:1789760739.983277:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724262139444 time 724303993718 state 0x0 ref 1 00000100:00100000:3.0:1789760739.983327:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03095180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495807488:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760739.983373:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23269376, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:3.0:1789760739.983383:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546323 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.983400:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760739.983404:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.983411:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546323 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.983425:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546323 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760740.030387:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972998 is committed 00002000:00010000:0.0:1789760740.034330:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23269376 -> 23273472 00010000:00010000:0.0:1789760740.034352:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546323 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.034367:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546323 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.034376:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546323 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.034383:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760740.034388:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546323 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.034397:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546323 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.034423:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546323 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.034431:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546323 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.034529:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495807488:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51200us (51496us total) trans 4294972998 rc 0/0 00000100:00100000:0.0:1789760740.034540:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5710 00000100:00100000:0.0:1789760740.034551:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724262139444 time 724303993718 state 0x2 ref 0 00000100:00100000:0.0:1789760740.040479:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760740.040619:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495807616 00000100:00100000:2.0:1789760740.040629:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495807616 00000100:00100000:2.0:1789760740.040682:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724262139444 time 724303993718 state 0x0 ref 1 00000100:00100000:2.0:1789760740.040697:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724304254134 time 724303993718 state 0x1 ref 1 00000100:00100000:2.0:1789760740.040727:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724304254134 time 724361444469 state 0x0 ref 1 00000100:00100000:2.0:1789760740.040739:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495807616:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760740.040787:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23273472, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:2.0:1789760740.040796:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454632a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.040812:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760740.040816:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.040823:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454632a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.040836:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454632a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760740.074829:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972999 is committed 00002000:00010000:3.0:1789760740.075238:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23273472 -> 23277568 00010000:00010000:3.0:1789760740.075255:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454632a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.075269:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454632a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.075302:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454632a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.075310:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760740.075314:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454632a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.075325:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454632a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.075345:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454632a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.075353:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454632a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.075443:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495807616:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34701us (34970us total) trans 4294972999 rc 0/0 00000100:00100000:3.0:1789760740.075456:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5711 00000100:00100000:3.0:1789760740.075467:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724304254134 time 724361444469 state 0x2 ref 0 00000100:00100000:3.0:1789760740.082870:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760740.083182:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495807744 00000100:00100000:2.0:1789760740.083191:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495807744 00000100:00100000:2.0:1789760740.083246:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724304254134 time 724361444469 state 0x0 ref 1 00000100:00100000:2.0:1789760740.083262:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724361704885 time 724361444469 state 0x1 ref 1 00000100:00100000:2.0:1789760740.083323:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724361704885 time 724404040095 state 0x0 ref 1 00000100:00100000:2.0:1789760740.083339:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495807744:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760740.083386:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23277568, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:2.0:1789760740.083395:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546331 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.083412:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760740.083417:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.083424:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546331 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.083437:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546331 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760740.114262:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973000 is committed 00002000:00010000:2.0:1789760740.114474:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23277568 -> 23281664 00010000:00010000:2.0:1789760740.114489:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546331 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.114501:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546331 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.114510:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546331 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.114516:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760740.114521:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546331 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.114529:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546331 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.114548:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546331 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.114555:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546331 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.114642:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495807744:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31300us (31776us total) trans 4294973000 rc 0/0 00000100:00100000:2.0:1789760740.114652:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5712 00000100:00100000:2.0:1789760740.114662:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724361704885 time 724404040095 state 0x2 ref 0 00000100:00100000:0.0:1789760740.118584:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760740.118905:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495807872 00000100:00100000:3.0:1789760740.118915:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495807872 00000100:00100000:3.0:1789760740.118967:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724361704885 time 724404040095 state 0x0 ref 1 00000100:00100000:3.0:1789760740.118982:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724404300511 time 724404040095 state 0x1 ref 1 00000100:00100000:3.0:1789760740.119012:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724404300511 time 724439729440 state 0x0 ref 1 00000100:00100000:3.0:1789760740.119025:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92901237ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495807872:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760740.119087:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23281664, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:3.0:1789760740.119096:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546338 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.119111:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760740.119115:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.119122:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546338 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.119136:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546338 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760740.150321:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23281664 -> 23285760 00010000:00010000:3.0:1789760740.150341:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546338 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.150354:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546338 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.150363:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546338 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.150371:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760740.150385:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546338 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.150394:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546338 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.150415:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546338 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.150423:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546338 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.150512:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92901237ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495807872:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31484us (31934us total) trans 4294973001 rc 0/0 00000100:00100000:3.0:1789760740.150522:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5713 00000100:00100000:3.0:1789760740.150531:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724404300511 time 724439729440 state 0x2 ref 0 00000001:00080000:1.0:1789760740.153978:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973001 is committed 00000100:00100000:1.0:1789760740.159361:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760740.159447:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495808000 00000100:00100000:3.0:1789760740.159457:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495808000 00000100:00100000:3.0:1789760740.159511:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724404300511 time 724439729440 state 0x0 ref 1 00000100:00100000:3.0:1789760740.159526:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724439989856 time 724439729440 state 0x1 ref 1 00000100:00100000:3.0:1789760740.159557:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724439989856 time 724480274276 state 0x0 ref 1 00000100:00100000:3.0:1789760740.159573:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495808000:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760740.159621:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23285760, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:3.0:1789760740.159630:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454633f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.159647:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760740.159651:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.159658:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454633f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.159671:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454633f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760740.184529:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23285760 -> 23289856 00010000:00010000:3.0:1789760740.184550:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454633f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.184564:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454633f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.184573:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454633f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.184579:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760740.184584:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454633f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.184593:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454633f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.184612:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454633f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.184619:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454633f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.184716:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495808000:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25140us (25360us total) trans 4294973002 rc 0/0 00000100:00100000:3.0:1789760740.184726:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5714 00000100:00100000:3.0:1789760740.184736:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724439989856 time 724480274276 state 0x2 ref 0 00000001:00080000:1.0:1789760740.184808:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973002 is committed 00000100:00100000:2.0:1789760740.188556:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760740.188650:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495808128 00000100:00100000:3.0:1789760740.188663:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495808128 00000100:00100000:3.0:1789760740.188715:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724439989856 time 724480274276 state 0x0 ref 1 00000100:00100000:3.0:1789760740.188731:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724480534692 time 724480274276 state 0x1 ref 1 00000100:00100000:3.0:1789760740.188763:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724480534692 time 724509479751 state 0x0 ref 1 00000100:00100000:3.0:1789760740.188776:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495808128:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760740.188822:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23289856, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:3.0:1789760740.188831:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546346 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.188848:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760740.188853:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.188859:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546346 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.188873:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546346 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.190407:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760740.220831:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973003 is committed 00002000:00010000:3.0:1789760740.220930:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23289856 -> 23293952 00010000:00010000:3.0:1789760740.220946:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546346 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.220959:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546346 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.220968:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546346 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.220975:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760740.220979:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546346 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.220988:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546346 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.221006:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546346 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.221013:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546346 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.221174:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495808128:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32396us (32622us total) trans 4294973003 rc 0/0 00000100:00100000:3.0:1789760740.221185:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5715 00000100:00100000:3.0:1789760740.221195:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724480534692 time 724509479751 state 0x2 ref 0 00000100:00100000:2.0:1789760740.224113:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760740.224235:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495808256 00000100:00100000:3.0:1789760740.224245:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495808256 00000100:00100000:3.0:1789760740.224330:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724480534692 time 724509479751 state 0x0 ref 1 00000100:00100000:3.0:1789760740.224344:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724509740167 time 724509479751 state 0x1 ref 1 00000100:00100000:3.0:1789760740.224378:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724509740167 time 724545088883 state 0x0 ref 1 00000100:00100000:3.0:1789760740.224390:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495808256:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760740.224438:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23293952, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:3.0:1789760740.224447:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454634d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.224463:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760740.224468:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.224474:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454634d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.224493:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454634d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.252829:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973004 is committed 00002000:00010000:3.0:1789760740.252933:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23293952 -> 23298048 00010000:00010000:3.0:1789760740.252950:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454634d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.252963:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454634d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.252972:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454634d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.252980:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760740.252986:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454634d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.252996:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454634d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.253016:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454634d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.253023:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454634d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.253169:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495808256:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28774us (29061us total) trans 4294973004 rc 0/0 00000100:00100000:3.0:1789760740.253180:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5716 00000100:00100000:3.0:1789760740.253190:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724509740167 time 724545088883 state 0x2 ref 0 00000100:00100000:1.0:1789760740.257608:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760740.257677:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495808384 00000100:00100000:3.0:1789760740.257687:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495808384 00000100:00100000:3.0:1789760740.257740:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724509740167 time 724545088883 state 0x0 ref 1 00000100:00100000:3.0:1789760740.257755:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724545349299 time 724545088883 state 0x1 ref 1 00000100:00100000:3.0:1789760740.257788:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724545349299 time 724578505092 state 0x0 ref 1 00000100:00100000:3.0:1789760740.257801:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495808384:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760740.257847:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23298048, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:3.0:1789760740.257856:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546354 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.257873:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760740.257878:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.257884:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546354 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.257898:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546354 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760740.297182:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973005 is committed 00002000:00010000:0.0:1789760740.297266:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23298048 -> 23302144 00010000:00010000:0.0:1789760740.297301:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546354 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.297314:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546354 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.297323:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546354 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.297330:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760740.297334:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546354 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.297343:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546354 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.297363:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546354 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.297370:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546354 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.297537:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495808384:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39734us (39933us total) trans 4294973005 rc 0/0 00000100:00100000:0.0:1789760740.297548:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5717 00000100:00100000:0.0:1789760740.297558:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724545349299 time 724578505092 state 0x2 ref 0 00000100:00100000:2.0:1789760740.300371:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760740.300474:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495808512 00000100:00100000:3.0:1789760740.300483:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495808512 00000100:00100000:3.0:1789760740.300534:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724545349299 time 724578505092 state 0x0 ref 1 00000100:00100000:3.0:1789760740.300549:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724578765508 time 724578505092 state 0x1 ref 1 00000100:00100000:3.0:1789760740.300607:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724578765508 time 724621323180 state 0x0 ref 1 00000100:00100000:3.0:1789760740.300619:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495808512:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760740.300669:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23302144, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:3.0:1789760740.300679:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454635b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.300694:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760740.300698:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.300704:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454635b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.300716:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454635b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760740.321697:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973006 is committed 00002000:00010000:0.0:1789760740.321830:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23302144 -> 23306240 00010000:00010000:0.0:1789760740.321844:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454635b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.321854:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454635b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.321860:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454635b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.321865:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760740.321869:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454635b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.321875:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454635b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.321890:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454635b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.321894:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454635b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.321992:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495808512:12345-192.168.203.9@tcp:4:dd.0 Request processed in 21372us (21626us total) trans 4294973006 rc 0/0 00000100:00100000:0.0:1789760740.322000:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5718 00000100:00100000:0.0:1789760740.322008:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724578765508 time 724621323180 state 0x2 ref 0 00000100:00100000:3.0:1789760740.325898:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760740.326008:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495808640 00000100:00100000:0.0:1789760740.326018:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495808640 00000100:00100000:0.0:1789760740.326185:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724578765508 time 724621323180 state 0x0 ref 1 00000100:00100000:0.0:1789760740.326200:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724621583596 time 724621323180 state 0x1 ref 1 00000100:00100000:0.0:1789760740.326236:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724621583596 time 724646953565 state 0x0 ref 1 00000100:00100000:0.0:1789760740.326249:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495808640:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760740.326326:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23306240, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:0.0:1789760740.326335:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546362 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.326351:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760740.326356:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.326362:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546362 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.326386:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546362 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760740.376001:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973007 is committed 00002000:00010000:1.0:1789760740.376219:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23306240 -> 23310336 00010000:00010000:1.0:1789760740.376237:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546362 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.376251:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546362 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.376261:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546362 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.376269:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760740.376274:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546362 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.376304:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546362 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.376332:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546362 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.376340:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546362 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760740.376585:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495808640:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50333us (50690us total) trans 4294973007 rc 0/0 00000100:00100000:1.0:1789760740.376597:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5719 00000100:00100000:1.0:1789760740.376608:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724621583596 time 724646953565 state 0x2 ref 0 00000100:00100000:1.0:1789760740.382267:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760740.382434:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495808768 00000100:00100000:0.0:1789760740.382447:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495808768 00000100:00100000:2.0:1789760740.382622:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724621583596 time 724646953565 state 0x0 ref 1 00000100:00100000:2.0:1789760740.382638:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724647213981 time 724646953565 state 0x1 ref 1 00000100:00100000:2.0:1789760740.382732:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724647213981 time 724703448024 state 0x0 ref 1 00000100:00100000:2.0:1789760740.382750:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495808768:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760740.382801:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23310336, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:2.0:1789760740.382811:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546369 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.382826:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760740.382831:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.382836:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546369 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.382848:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546369 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760740.419109:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973008 is committed 00002000:00010000:0.0:1789760740.419789:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23310336 -> 23314432 00010000:00010000:0.0:1789760740.419808:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546369 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.419820:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546369 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.419853:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546369 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.419861:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760740.419867:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546369 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.419881:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546369 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.419901:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546369 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.419909:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546369 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.421828:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495808768:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39067us (39557us total) trans 4294973008 rc 0/0 00000100:00100000:0.0:1789760740.421842:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5720 00000100:00100000:0.0:1789760740.421854:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724647213981 time 724703448024 state 0x2 ref 0 00000100:00100000:3.0:1789760740.424854:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760740.424990:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495808896 00000100:00100000:0.0:1789760740.424999:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495808896 00000100:00100000:0.0:1789760740.425112:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724647213981 time 724703448024 state 0x0 ref 1 00000100:00100000:0.0:1789760740.425129:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724703708440 time 724703448024 state 0x1 ref 1 00000100:00100000:0.0:1789760740.425160:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724703708440 time 724745875661 state 0x0 ref 1 00000100:00100000:0.0:1789760740.425172:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495808896:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760740.425224:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23314432, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:0.0:1789760740.425233:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546370 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.425249:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760740.425253:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.425260:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546370 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.425273:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546370 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760740.460113:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23314432 -> 23318528 00010000:00010000:1.0:1789760740.460134:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546370 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.460147:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546370 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.460156:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546370 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.460163:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760740.460168:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546370 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.460177:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546370 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.460200:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546370 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.460208:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546370 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760740.460371:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495808896:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35195us (35520us total) trans 4294973009 rc 0/0 00000100:00100000:1.0:1789760740.460382:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5721 00000100:00100000:1.0:1789760740.460394:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724703708440 time 724745875661 state 0x2 ref 0 00000001:00080000:3.0:1789760740.460645:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973009 is committed 00000100:00100000:3.0:1789760740.465176:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760740.465609:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495809024 00000100:00100000:1.0:1789760740.465620:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495809024 00000100:00100000:2.0:1789760740.465747:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724703708440 time 724745875661 state 0x0 ref 1 00000100:00100000:2.0:1789760740.465762:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724746136077 time 724745875661 state 0x1 ref 1 00000100:00100000:2.0:1789760740.465827:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724746136077 time 724786543433 state 0x0 ref 1 00000100:00100000:2.0:1789760740.465841:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495809024:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760740.465897:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23318528, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:2.0:1789760740.465907:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546377 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.465924:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760740.465929:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.465934:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546377 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.465945:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546377 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760740.508972:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973010 is committed 00002000:00010000:3.0:1789760740.509088:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23318528 -> 23322624 00010000:00010000:3.0:1789760740.509103:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546377 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.509116:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546377 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.509126:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546377 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.509133:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760740.509138:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546377 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.509147:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546377 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.509166:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546377 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.509174:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546377 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.509336:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495809024:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43493us (44165us total) trans 4294973010 rc 0/0 00000100:00100000:3.0:1789760740.509347:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5722 00000100:00100000:3.0:1789760740.509357:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724746136077 time 724786543433 state 0x2 ref 0 00000100:00100000:1.0:1789760740.512505:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760740.512594:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495809152 00000100:00100000:0.0:1789760740.512604:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495809152 00000100:00100000:0.0:1789760740.512655:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724746136077 time 724786543433 state 0x0 ref 1 00000100:00100000:0.0:1789760740.512671:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724786803849 time 724786543433 state 0x1 ref 1 00000100:00100000:0.0:1789760740.512701:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724786803849 time 724833418201 state 0x0 ref 1 00000100:00100000:0.0:1789760740.512713:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495809152:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760740.512768:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23322624, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:0.0:1789760740.512778:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454637e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.512793:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760740.512797:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.512804:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454637e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.512819:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454637e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760740.539825:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973011 is committed 00002000:00010000:0.0:1789760740.539950:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23322624 -> 23326720 00010000:00010000:0.0:1789760740.539966:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454637e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.539979:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454637e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.539987:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454637e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.539995:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760740.539999:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454637e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.540010:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454637e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.540104:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454637e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.540113:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454637e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.540213:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495809152:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27496us (27712us total) trans 4294973011 rc 0/0 00000100:00100000:0.0:1789760740.540223:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5723 00000100:00100000:0.0:1789760740.540233:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724786803849 time 724833418201 state 0x2 ref 0 00000100:00100000:0.0:1789760740.548856:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760740.548958:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495809280 00000100:00100000:1.0:1789760740.548968:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495809280 00000100:00100000:1.0:1789760740.549023:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724786803849 time 724833418201 state 0x0 ref 1 00000100:00100000:1.0:1789760740.549043:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724833678617 time 724833418201 state 0x1 ref 1 00000100:00100000:1.0:1789760740.549082:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724833678617 time 724869799501 state 0x0 ref 1 00000100:00100000:1.0:1789760740.549093:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495809280:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760740.549136:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23326720, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:1.0:1789760740.549145:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546385 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.549161:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760740.549166:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760740.549172:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546385 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.549187:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546385 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760740.569438:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23326720 -> 23330816 00000001:00080000:3.0:1789760740.569457:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973012 is committed 00010000:00010000:1.0:1789760740.569459:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546385 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.569472:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546385 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.569481:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546385 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.569489:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760740.569505:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546385 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.569514:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546385 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.569537:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546385 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.569545:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546385 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760740.569637:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495809280:12345-192.168.203.9@tcp:4:dd.0 Request processed in 20540us (20787us total) trans 4294973012 rc 0/0 00000100:00100000:1.0:1789760740.569647:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5724 00000100:00100000:1.0:1789760740.569657:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724833678617 time 724869799501 state 0x2 ref 0 00000100:00100000:2.0:1789760740.572544:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760740.572753:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495809408 00000100:00100000:3.0:1789760740.572764:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495809408 00000100:00100000:3.0:1789760740.572819:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724833678617 time 724869799501 state 0x0 ref 1 00000100:00100000:3.0:1789760740.572835:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724870059917 time 724869799501 state 0x1 ref 1 00000100:00100000:3.0:1789760740.572900:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724870059917 time 724893615644 state 0x0 ref 1 00000100:00100000:3.0:1789760740.572913:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903639ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495809408:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760740.572958:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23330816, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:3.0:1789760740.572966:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454638c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.572982:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760740.572987:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.572992:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454638c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.573005:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454638c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760740.596932:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973013 is committed 00002000:00010000:2.0:1789760740.597065:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23330816 -> 23334912 00010000:00010000:2.0:1789760740.597091:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454638c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.597105:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454638c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.597334:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454638c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.597342:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760740.597347:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454638c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.597356:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454638c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.597442:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454638c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.597458:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454638c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.597549:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903639ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495809408:12345-192.168.203.9@tcp:4:dd.0 Request processed in 24632us (25009us total) trans 4294973013 rc 0/0 00000100:00100000:2.0:1789760740.597559:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5725 00000100:00100000:2.0:1789760740.597569:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724870059917 time 724893615644 state 0x2 ref 0 00000100:00100000:2.0:1789760740.602754:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760740.602908:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495809536 00000100:00100000:0.0:1789760740.602917:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495809536 00000100:00100000:0.0:1789760740.602970:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724870059917 time 724893615644 state 0x0 ref 1 00000100:00100000:0.0:1789760740.602987:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724893876060 time 724893615644 state 0x1 ref 1 00000100:00100000:0.0:1789760740.603012:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724893876060 time 724923729686 state 0x0 ref 1 00000100:00100000:0.0:1789760740.603025:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495809536:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760740.603076:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23334912, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:0.0:1789760740.603093:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546393 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.603107:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760740.603112:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.603118:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546393 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.603131:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546393 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.640916:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff929040814000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520306944:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760740.640988:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760740.641183:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520306944 00000100:00100000:0.0:1789760740.641191:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520306944 00000100:00100000:0.0:1789760740.641248:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 199 00000100:00100000:0.0:1789760740.641259:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520306944:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789760740.641303:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528547840 pending 1703936 free 3834642432 avail 3791650816 00000100:00100000:0.0:1789760740.641351:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520306944:12345-0@lo:13:osp-pre-0-0.0 Request processed in 94us (362us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760740.641359:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 199 00000100:00100000:0.0:1789760740.641408:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff929040814000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520306944:0@lo:13:osp-pre-0-0.0 00000100:00080000:1.0:1789760740.641700:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789760740.650206:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23334912 -> 23339008 00010000:00010000:1.0:1789760740.650225:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546393 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.650238:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546393 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.650246:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546393 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.650253:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760740.650257:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546393 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.650268:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546393 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.650307:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546393 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.650316:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546393 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760740.650414:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495809536:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47386us (47664us total) trans 4294973014 rc 0/0 00000100:00100000:1.0:1789760740.650425:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5726 00000100:00100000:1.0:1789760740.650436:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724893876060 time 724923729686 state 0x2 ref 0 00000001:00080000:3.0:1789760740.650514:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973014 is committed 00000100:00100000:0.0:1789760740.653993:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760740.654057:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495809664 00000100:00100000:1.0:1789760740.654066:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495809664 00000100:00100000:1.0:1789760740.654309:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724893876060 time 724923729686 state 0x0 ref 1 00000100:00100000:1.0:1789760740.654327:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724923990102 time 724923729686 state 0x1 ref 1 00000100:00100000:1.0:1789760740.654359:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724923990102 time 724975075820 state 0x0 ref 1 00000100:00100000:1.0:1789760740.654475:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495809664:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760740.654525:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23339008, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:1.0:1789760740.654533:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454639a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.654549:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760740.654553:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760740.654559:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454639a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.654572:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454639a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760740.682103:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973015 is committed 00002000:00010000:3.0:1789760740.682126:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23339008 -> 23343104 00002000:00010000:3.0:1789760740.682132:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760739 -> 1789760740 00002000:00010000:3.0:1789760740.682136:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760739 -> 1789760740 00010000:00010000:3.0:1789760740.682150:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454639a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.682164:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454639a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.682173:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454639a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.682181:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760740.682185:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454639a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.682195:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454639a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.682217:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454639a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.682225:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454639a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.682420:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495809664:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28046us (28430us total) trans 4294973015 rc 0/0 00000100:00100000:3.0:1789760740.682431:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5727 00000100:00100000:3.0:1789760740.682441:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724923990102 time 724975075820 state 0x2 ref 0 00000100:00100000:2.0:1789760740.689749:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760740.689940:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495809792 00000100:00100000:3.0:1789760740.689950:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495809792 00000100:00100000:3.0:1789760740.690010:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724923990102 time 724975075820 state 0x0 ref 1 00000100:00100000:3.0:1789760740.690032:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724975336236 time 724975075820 state 0x1 ref 1 00000100:00100000:3.0:1789760740.690066:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724975336236 time 725010782489 state 0x0 ref 1 00000100:00100000:3.0:1789760740.690078:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495809792:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760740.690330:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23343104, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:3.0:1789760740.690340:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545463a1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.690356:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760740.690360:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.690367:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545463a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.690381:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545463a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760740.732371:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973016 is committed 00002000:00010000:3.0:1789760740.732852:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23343104 -> 23347200 00010000:00010000:3.0:1789760740.732869:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545463a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.732882:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545463a1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.732891:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545463a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.732900:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760740.732905:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545463a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.732915:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545463a1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.732935:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545463a1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.732943:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545463a1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.733137:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495809792:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43055us (43390us total) trans 4294973016 rc 0/0 00000100:00100000:3.0:1789760740.733148:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5728 00000100:00100000:3.0:1789760740.733168:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724975336236 time 725010782489 state 0x2 ref 0 00000100:00100000:2.0:1789760740.736908:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760740.737006:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495809920 00000100:00100000:1.0:1789760740.737017:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495809920 00000100:00100000:1.0:1789760740.737092:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 724975336236 time 725010782489 state 0x0 ref 1 00000100:00100000:1.0:1789760740.737116:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725011042905 time 725010782489 state 0x1 ref 1 00000100:00100000:1.0:1789760740.737146:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725011042905 time 725057862971 state 0x0 ref 1 00000100:00100000:1.0:1789760740.737167:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495809920:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760740.737216:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23347200, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:1.0:1789760740.737226:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545463a8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.737241:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760740.737246:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760740.737253:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545463a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.737267:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545463a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760740.765732:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23347200 -> 23351296 00010000:00010000:1.0:1789760740.765753:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545463a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.765767:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545463a8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.765777:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545463a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.765785:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760740.765790:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545463a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.765799:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545463a8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.765817:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545463a8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.765825:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545463a8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760740.765915:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495809920:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28746us (29011us total) trans 4294973017 rc 0/0 00000100:00100000:1.0:1789760740.765926:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5729 00000100:00100000:1.0:1789760740.765935:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725011042905 time 725057862971 state 0x2 ref 0 00000001:00080000:1.0:1789760740.765970:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973017 is committed 00000100:00100000:0.0:1789760740.773933:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760740.774162:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495810048 00000100:00100000:3.0:1789760740.774173:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495810048 00000100:00100000:2.0:1789760740.774399:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725011042905 time 725057862971 state 0x0 ref 1 00000100:00100000:2.0:1789760740.774417:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725058123387 time 725057862971 state 0x1 ref 1 00000100:00100000:2.0:1789760740.774446:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725058123387 time 725095162798 state 0x0 ref 1 00000100:00100000:2.0:1789760740.774459:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495810048:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760740.774506:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23351296, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:2.0:1789760740.774515:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545463af lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.774762:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760740.774768:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.774774:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545463af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.774790:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545463af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760740.810304:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973018 is committed 00002000:00010000:3.0:1789760740.810482:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23351296 -> 23355392 00010000:00010000:3.0:1789760740.810496:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545463af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.810512:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545463af lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.810520:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545463af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.810526:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760740.810531:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545463af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.810539:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545463af lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.810558:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545463af lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.810566:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545463af lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.811193:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495810048:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36731us (37264us total) trans 4294973018 rc 0/0 00000100:00100000:3.0:1789760740.811204:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5730 00000100:00100000:3.0:1789760740.811214:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725058123387 time 725095162798 state 0x2 ref 0 00000100:00100000:1.0:1789760740.816414:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760740.816625:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495810176 00000100:00100000:3.0:1789760740.816636:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495810176 00000100:00100000:3.0:1789760740.816692:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725058123387 time 725095162798 state 0x0 ref 1 00000100:00100000:3.0:1789760740.816708:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725095423214 time 725095162798 state 0x1 ref 1 00000100:00100000:3.0:1789760740.816743:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725095423214 time 725137459440 state 0x0 ref 1 00000100:00100000:3.0:1789760740.816756:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495810176:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760740.816804:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23355392, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:3.0:1789760740.816814:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545463b6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.816830:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760740.816834:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.816841:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545463b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.816855:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545463b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.858959:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973019 is committed 00002000:00010000:3.0:1789760740.859095:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23355392 -> 23359488 00010000:00010000:3.0:1789760740.859111:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545463b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.859124:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545463b6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.859348:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545463b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.859355:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760740.859361:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545463b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.859443:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545463b6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.859463:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545463b6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.859472:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545463b6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.859819:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495810176:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43059us (43411us total) trans 4294973019 rc 0/0 00000100:00100000:0.0:1789760740.859831:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5731 00000100:00100000:0.0:1789760740.859842:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725095423214 time 725137459440 state 0x2 ref 0 00000100:00100000:0.0:1789760740.864658:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760740.864742:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495810304 00000100:00100000:1.0:1789760740.864751:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495810304 00000100:00100000:1.0:1789760740.864802:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725095423214 time 725137459440 state 0x0 ref 1 00000100:00100000:1.0:1789760740.864817:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725137719856 time 725137459440 state 0x1 ref 1 00000100:00100000:1.0:1789760740.864846:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725137719856 time 725185562800 state 0x0 ref 1 00000100:00100000:1.0:1789760740.864858:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495810304:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760740.864905:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23359488, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:1.0:1789760740.864913:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545463bd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.864929:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760740.864933:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760740.864939:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545463bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.864957:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545463bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760740.917361:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973020 is committed 00002000:00010000:2.0:1789760740.917464:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23359488 -> 23363584 00010000:00010000:2.0:1789760740.917479:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545463bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.917491:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545463bd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.917499:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545463bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.917506:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760740.917511:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545463bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.917519:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545463bd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.917538:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545463bd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.917545:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545463bd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.917791:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495810304:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52930us (53136us total) trans 4294973020 rc 0/0 00000100:00100000:2.0:1789760740.917803:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5732 00000100:00100000:2.0:1789760740.917812:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725137719856 time 725185562800 state 0x2 ref 0 00000100:00100000:3.0:1789760740.931137:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760740.931246:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495810432 00000100:00100000:2.0:1789760740.931256:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495810432 00000100:00100000:2.0:1789760740.931342:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725137719856 time 725185562800 state 0x0 ref 1 00000100:00100000:2.0:1789760740.931357:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725185823216 time 725185562800 state 0x1 ref 1 00000100:00100000:2.0:1789760740.931379:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725185823216 time 725252096296 state 0x0 ref 1 00000100:00100000:2.0:1789760740.931392:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495810432:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760740.931439:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23363584, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:2.0:1789760740.931448:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545463c4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.931463:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760740.931467:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.931474:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545463c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.931488:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545463c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760740.989789:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23363584 -> 23367680 00010000:00010000:2.0:1789760740.989808:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545463c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.989821:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545463c4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.989829:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545463c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.989837:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760740.989842:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545463c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.989852:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545463c4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.989871:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545463c4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.989878:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545463c4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.989976:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495810432:12345-192.168.203.9@tcp:4:dd.0 Request processed in 58582us (58843us total) trans 4294973021 rc 0/0 00000100:00100000:2.0:1789760740.989991:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5733 00000100:00100000:2.0:1789760740.990001:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725185823216 time 725252096296 state 0x2 ref 0 00000001:00080000:0.0:1789760740.990832:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973021 is committed 00000100:00100000:0.0:1789760740.993745:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760740.993827:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495810560 00000100:00100000:2.0:1789760740.993838:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495810560 00000100:00100000:2.0:1789760740.993898:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725185823216 time 725252096296 state 0x0 ref 1 00000100:00100000:2.0:1789760740.993915:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725252356712 time 725252096296 state 0x1 ref 1 00000100:00100000:2.0:1789760740.993944:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725252356712 time 725314661518 state 0x0 ref 1 00000100:00100000:2.0:1789760740.993958:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495810560:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760740.994008:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23367680, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:2.0:1789760740.994018:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545463cb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.994036:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760740.994041:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.994056:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545463cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.994069:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545463cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760741.038246:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973022 is committed 00002000:00010000:3.0:1789760741.038304:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23367680 -> 23371776 00010000:00010000:3.0:1789760741.038321:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545463cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.038334:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545463cb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.038343:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545463cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.038351:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760741.038356:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545463cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.038366:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545463cb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.038385:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545463cb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.038394:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545463cb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.038486:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495810560:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44526us (44745us total) trans 4294973022 rc 0/0 00000100:00100000:3.0:1789760741.038497:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5734 00000100:00100000:3.0:1789760741.038507:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725252356712 time 725314661518 state 0x2 ref 0 00000100:00100000:3.0:1789760741.045235:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760741.046117:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495810688 00000100:00100000:2.0:1789760741.046128:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495810688 00000100:00100000:2.0:1789760741.046201:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725252356712 time 725314661518 state 0x0 ref 1 00000100:00100000:2.0:1789760741.046221:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725314921934 time 725314661518 state 0x1 ref 1 00000100:00100000:2.0:1789760741.046251:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725314921934 time 725366967623 state 0x0 ref 1 00000100:00100000:2.0:1789760741.046263:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495810688:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760741.046345:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23371776, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:2.0:1789760741.046354:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545463d2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.046370:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760741.046375:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.046381:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545463d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.046395:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545463d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760741.046987:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789760741.074971:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23371776 -> 23375872 00010000:00010000:3.0:1789760741.075002:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545463d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.075016:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545463d2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.075039:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545463d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.075056:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760741.075061:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545463d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.075069:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545463d2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.075089:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545463d2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.075097:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545463d2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760741.075181:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973023 is committed 00000100:00100000:3.0:1789760741.075192:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495810688:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28927us (29962us total) trans 4294973023 rc 0/0 00000100:00100000:3.0:1789760741.075202:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5735 00000100:00100000:3.0:1789760741.075211:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725314921934 time 725366967623 state 0x2 ref 0 00000100:00100000:0.0:1789760741.079650:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760741.079729:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495810816 00000100:00100000:2.0:1789760741.079739:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495810816 00000100:00100000:2.0:1789760741.079790:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725314921934 time 725366967623 state 0x0 ref 1 00000100:00100000:2.0:1789760741.079806:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725367228039 time 725366967623 state 0x1 ref 1 00000100:00100000:2.0:1789760741.079836:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725367228039 time 725400553152 state 0x0 ref 1 00000100:00100000:2.0:1789760741.079848:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495810816:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760741.079895:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23375872, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:2.0:1789760741.079904:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545463d9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.079920:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760741.079925:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.079932:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545463d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.079946:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545463d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760741.110591:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973024 is committed 00002000:00010000:3.0:1789760741.110728:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23375872 -> 23379968 00010000:00010000:3.0:1789760741.110748:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545463d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.110771:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545463d9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.110783:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545463d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.110790:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760741.110795:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545463d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.110803:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545463d9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.110823:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545463d9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.110832:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545463d9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.110924:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495810816:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31072us (31277us total) trans 4294973024 rc 0/0 00000100:00100000:3.0:1789760741.110937:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5736 00000100:00100000:3.0:1789760741.110947:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725367228039 time 725400553152 state 0x2 ref 0 00000100:00100000:3.0:1789760741.115627:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760741.115717:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495810944 00000100:00100000:0.0:1789760741.115727:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495810944 00000100:00100000:0.0:1789760741.115782:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725367228039 time 725400553152 state 0x0 ref 1 00000100:00100000:0.0:1789760741.115798:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725400813568 time 725400553152 state 0x1 ref 1 00000100:00100000:0.0:1789760741.115830:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725400813568 time 725436546515 state 0x0 ref 1 00000100:00100000:0.0:1789760741.115843:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495810944:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760741.115890:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23379968, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:0.0:1789760741.115900:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545463e0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.115916:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760741.115921:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.115928:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545463e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.115942:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545463e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760741.156018:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23379968 -> 23384064 00010000:00010000:0.0:1789760741.156112:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545463e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.156131:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545463e0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.156140:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545463e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.156148:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760741.156153:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545463e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.156167:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545463e0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760741.156197:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973025 is committed 00010000:00010000:0.0:1789760741.156251:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545463e0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.156260:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545463e0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.156556:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495810944:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40710us (40934us total) trans 4294973025 rc 0/0 00000100:00100000:0.0:1789760741.156566:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5737 00000100:00100000:0.0:1789760741.156576:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725400813568 time 725436546515 state 0x2 ref 0 00000100:00100000:0.0:1789760741.163110:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760741.163178:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495811072 00000100:00100000:3.0:1789760741.163187:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495811072 00000100:00100000:3.0:1789760741.163255:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725400813568 time 725436546515 state 0x0 ref 1 00000100:00100000:3.0:1789760741.163270:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725436806931 time 725436546515 state 0x1 ref 1 00000100:00100000:3.0:1789760741.163325:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725436806931 time 725484041344 state 0x0 ref 1 00000100:00100000:3.0:1789760741.163338:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495811072:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760741.163392:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23384064, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:3.0:1789760741.163402:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463e7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.163418:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760741.163423:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.163429:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.163441:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760741.195910:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973026 is committed 00002000:00010000:3.0:1789760741.196141:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23384064 -> 23388160 00010000:00010000:3.0:1789760741.196154:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.196163:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463e7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.196170:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.196178:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760741.196182:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.196190:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463e7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.196214:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463e7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.196222:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463e7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.196353:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495811072:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33011us (33245us total) trans 4294973026 rc 0/0 00000100:00100000:3.0:1789760741.196363:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5738 00000100:00100000:3.0:1789760741.196372:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725436806931 time 725484041344 state 0x2 ref 0 00000100:00100000:1.0:1789760741.201837:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760741.201937:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495811200 00000100:00100000:3.0:1789760741.201947:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495811200 00000100:00100000:3.0:1789760741.202001:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725436806931 time 725484041344 state 0x0 ref 1 00000100:00100000:3.0:1789760741.202017:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725484301760 time 725484041344 state 0x1 ref 1 00000100:00100000:3.0:1789760741.202114:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725484301760 time 725522830353 state 0x0 ref 1 00000100:00100000:3.0:1789760741.202128:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495811200:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760741.202245:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23388160, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:3.0:1789760741.202255:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463ee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.202271:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760741.202275:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.202632:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.202649:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.214683:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760741.232332:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23388160 -> 23392256 00010000:00010000:3.0:1789760741.232352:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.232439:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463ee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.232449:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.232456:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760741.232462:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.232472:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463ee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.232492:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463ee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.232500:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463ee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.232589:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495811200:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30459us (30756us total) trans 4294973027 rc 0/0 00000100:00100000:3.0:1789760741.232600:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5739 00000100:00100000:3.0:1789760741.232610:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725484301760 time 725522830353 state 0x2 ref 0 00000001:00080000:0.0:1789760741.232693:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973027 is committed 00000100:00100000:1.0:1789760741.236523:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760741.236669:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495811328 00000100:00100000:0.0:1789760741.236680:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495811328 00000100:00100000:0.0:1789760741.236739:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725484301760 time 725522830353 state 0x0 ref 1 00000100:00100000:0.0:1789760741.236755:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725523090769 time 725522830353 state 0x1 ref 1 00000100:00100000:0.0:1789760741.236785:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725523090769 time 725557502329 state 0x0 ref 1 00000100:00100000:0.0:1789760741.236797:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495811328:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760741.236844:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23392256, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:0.0:1789760741.236852:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545463f5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.236868:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760741.236872:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.236878:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545463f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.236891:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545463f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760741.267544:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973028 is committed 00002000:00010000:0.0:1789760741.267649:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23392256 -> 23396352 00010000:00010000:0.0:1789760741.267665:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545463f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.267684:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545463f5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.267693:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545463f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.267700:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760741.267705:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545463f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.267714:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545463f5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.267733:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545463f5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.267741:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545463f5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.267838:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495811328:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31038us (31320us total) trans 4294973028 rc 0/0 00000100:00100000:0.0:1789760741.267849:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5740 00000100:00100000:0.0:1789760741.267859:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725523090769 time 725557502329 state 0x2 ref 0 00000100:00100000:1.0:1789760741.272443:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760741.272519:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495811456 00000100:00100000:3.0:1789760741.272529:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495811456 00000100:00100000:3.0:1789760741.272580:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725523090769 time 725557502329 state 0x0 ref 1 00000100:00100000:3.0:1789760741.272596:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725557762745 time 725557502329 state 0x1 ref 1 00000100:00100000:3.0:1789760741.272671:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725557762745 time 725593386838 state 0x0 ref 1 00000100:00100000:3.0:1789760741.272684:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495811456:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760741.272735:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23396352, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:3.0:1789760741.272744:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463fc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.272760:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760741.272765:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.272771:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.272785:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760741.309384:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23396352 -> 23400448 00010000:00010000:3.0:1789760741.309403:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.309416:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463fc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.309425:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.309431:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760741.309437:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.309446:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.309466:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463fc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.309473:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545463fc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.309562:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495811456:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36875us (37284us total) trans 4294973029 rc 0/0 00000100:00100000:3.0:1789760741.309571:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5741 00000100:00100000:3.0:1789760741.309581:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725557762745 time 725593386838 state 0x2 ref 0 00000100:00100000:2.0:1789760741.313126:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:0.0:1789760741.313322:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973029 is committed 00000100:00100000:3.0:1789760741.313530:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495811584 00000100:00100000:3.0:1789760741.313540:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495811584 00000100:00100000:3.0:1789760741.313596:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725557762745 time 725593386838 state 0x0 ref 1 00000100:00100000:3.0:1789760741.313681:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725593647254 time 725593386838 state 0x1 ref 1 00000100:00100000:3.0:1789760741.313747:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725593647254 time 725634463089 state 0x0 ref 1 00000100:00100000:3.0:1789760741.313759:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495811584:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760741.313803:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23400448, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:3.0:1789760741.313812:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546403 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.313827:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760741.313832:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.313838:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546403 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.313852:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546403 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.341687:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973030 is committed 00002000:00010000:0.0:1789760741.341799:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23400448 -> 23404544 00010000:00010000:0.0:1789760741.341814:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546403 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.341827:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546403 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.341836:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546403 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.341843:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760741.341848:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546403 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.341857:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546403 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.341878:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546403 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.341886:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546403 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.341990:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495811584:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28226us (28868us total) trans 4294973030 rc 0/0 00000100:00100000:0.0:1789760741.342001:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5742 00000100:00100000:0.0:1789760741.342013:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725593647254 time 725634463089 state 0x2 ref 0 00000100:00100000:1.0:1789760741.346239:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760741.346356:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495811712 00000100:00100000:3.0:1789760741.346375:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495811712 00000100:00100000:3.0:1789760741.346426:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725593647254 time 725634463089 state 0x0 ref 1 00000100:00100000:3.0:1789760741.346440:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725634723505 time 725634463089 state 0x1 ref 1 00000100:00100000:3.0:1789760741.346474:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725634723505 time 725667190463 state 0x0 ref 1 00000100:00100000:3.0:1789760741.346487:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+6:2357:x1876699495811712:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760741.346535:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23404544, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:3.0:1789760741.346543:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454640a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.346559:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760741.346564:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.346570:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454640a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.346583:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454640a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760741.369917:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973031 is committed 00002000:00010000:1.0:1789760741.370051:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23404544 -> 23408640 00010000:00010000:1.0:1789760741.370066:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454640a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.370079:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454640a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.370089:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454640a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.370096:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760741.370101:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454640a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.370110:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454640a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.370134:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454640a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.370142:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454640a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760741.370246:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495811712:12345-192.168.203.9@tcp:4:dd.0 Request processed in 23756us (24076us total) trans 4294973031 rc 0/0 00000100:00100000:1.0:1789760741.370257:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5743 00000100:00100000:1.0:1789760741.370268:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725634723505 time 725667190463 state 0x2 ref 0 00000100:00100000:1.0:1789760741.377688:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760741.377787:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495811840 00000100:00100000:3.0:1789760741.377941:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495811840 00000100:00100000:3.0:1789760741.378000:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725634723505 time 725667190463 state 0x0 ref 1 00000100:00100000:3.0:1789760741.378016:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725667450879 time 725667190463 state 0x1 ref 1 00000100:00100000:3.0:1789760741.378068:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725667450879 time 725698784388 state 0x0 ref 1 00000100:00100000:3.0:1789760741.378081:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495811840:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760741.378155:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23408640, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:3.0:1789760741.378164:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546411 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.378180:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760741.378184:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.378190:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546411 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.378202:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546411 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.412242:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973032 is committed 00002000:00010000:2.0:1789760741.412493:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23408640 -> 23412736 00010000:00010000:2.0:1789760741.412510:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546411 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.412522:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546411 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.412531:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546411 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.412540:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760741.412611:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546411 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.412625:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546411 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.412644:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546411 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.412651:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546411 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760741.412748:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495811840:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34664us (35064us total) trans 4294973032 rc 0/0 00000100:00100000:2.0:1789760741.412758:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5744 00000100:00100000:2.0:1789760741.412768:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725667450879 time 725698784388 state 0x2 ref 0 00000100:00100000:2.0:1789760741.416475:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760741.416632:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495811968 00000100:00100000:0.0:1789760741.416643:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495811968 00000100:00100000:0.0:1789760741.416697:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725667450879 time 725698784388 state 0x0 ref 1 00000100:00100000:0.0:1789760741.416714:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725699044804 time 725698784388 state 0x1 ref 1 00000100:00100000:0.0:1789760741.416747:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725699044804 time 725737463474 state 0x0 ref 1 00000100:00100000:0.0:1789760741.416760:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495811968:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760741.416806:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23412736, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:0.0:1789760741.416815:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546418 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.416830:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760741.416834:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.416840:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546418 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.416854:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546418 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760741.460480:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973033 is committed 00002000:00010000:1.0:1789760741.460567:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23412736 -> 23416832 00010000:00010000:1.0:1789760741.460583:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546418 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.460667:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546418 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.460678:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546418 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.460686:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760741.460691:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546418 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.460702:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546418 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.460720:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546418 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.460729:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546418 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760741.460917:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495811968:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44154us (44444us total) trans 4294973033 rc 0/0 00000100:00100000:1.0:1789760741.460928:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5745 00000100:00100000:1.0:1789760741.460938:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725699044804 time 725737463474 state 0x2 ref 0 00000100:00100000:1.0:1789760741.471355:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760741.471470:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495812096 00000100:00100000:3.0:1789760741.471480:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495812096 00000100:00100000:3.0:1789760741.471535:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725699044804 time 725737463474 state 0x0 ref 1 00000100:00100000:3.0:1789760741.471555:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725737723890 time 725737463474 state 0x1 ref 1 00000100:00100000:3.0:1789760741.471584:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725737723890 time 725792300971 state 0x0 ref 1 00000100:00100000:3.0:1789760741.471597:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495812096:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760741.471644:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23416832, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:3.0:1789760741.471654:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454641f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.471669:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760741.471674:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.471680:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454641f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.471693:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454641f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760741.519916:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23416832 -> 23420928 00010000:00010000:3.0:1789760741.519937:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454641f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.519951:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454641f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.519960:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454641f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.519974:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760741.519979:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454641f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.519989:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454641f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.520009:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454641f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.520017:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454641f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.520117:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495812096:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48518us (48766us total) trans 4294973034 rc 0/0 00000100:00100000:3.0:1789760741.520128:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5746 00000100:00100000:3.0:1789760741.520138:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725737723890 time 725792300971 state 0x2 ref 0 00000001:00080000:1.0:1789760741.520866:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973034 is committed 00000100:00100000:0.0:1789760741.526591:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760741.526858:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495812224 00000100:00100000:2.0:1789760741.526869:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495812224 00000100:00100000:2.0:1789760741.526923:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725737723890 time 725792300971 state 0x0 ref 1 00000100:00100000:2.0:1789760741.526939:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725792561387 time 725792300971 state 0x1 ref 1 00000100:00100000:2.0:1789760741.527004:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725792561387 time 725847719748 state 0x0 ref 1 00000100:00100000:2.0:1789760741.527017:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495812224:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760741.527084:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23420928, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:2.0:1789760741.527103:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546426 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.527121:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760741.527125:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.527131:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546426 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.527144:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546426 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760741.570889:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23420928 -> 23425024 00010000:00010000:3.0:1789760741.570908:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546426 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.570921:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546426 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.570935:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546426 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.570942:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760741.570947:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546426 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.570956:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546426 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.570972:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546426 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.570979:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546426 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.571078:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495812224:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44056us (44490us total) trans 4294973035 rc 0/0 00000100:00100000:3.0:1789760741.571088:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5747 00000100:00100000:3.0:1789760741.571098:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725792561387 time 725847719748 state 0x2 ref 0 00000001:00080000:1.0:1789760741.571731:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973035 is committed 00000100:00100000:2.0:1789760741.575261:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760741.576580:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495812352 00000100:00100000:1.0:1789760741.576593:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495812352 00000100:00100000:2.0:1789760741.576801:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725792561387 time 725847719748 state 0x0 ref 1 00000100:00100000:2.0:1789760741.576817:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725847980164 time 725847719748 state 0x1 ref 1 00000100:00100000:2.0:1789760741.576973:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725847980164 time 725897687744 state 0x0 ref 1 00000100:00100000:2.0:1789760741.576986:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04290700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495812352:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760741.577101:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23425024, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:2.0:1789760741.577111:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454642d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.577127:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760741.577131:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.577138:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454642d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.577157:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454642d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760741.627728:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23425024 -> 23429120 00010000:00010000:3.0:1789760741.627750:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454642d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.627770:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454642d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.627779:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454642d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.627787:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760741.627792:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454642d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.627803:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454642d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.627826:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454642d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.627834:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454642d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.627842:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973036 is committed 00000100:00100000:3.0:1789760741.627931:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04290700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495812352:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50942us (52674us total) trans 4294973036 rc 0/0 00000100:00100000:3.0:1789760741.627942:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5748 00000100:00100000:3.0:1789760741.627952:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725847980164 time 725897687744 state 0x2 ref 0 00000100:00100000:0.0:1789760741.633007:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760741.633194:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495812480 00000100:00100000:3.0:1789760741.633204:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495812480 00000100:00100000:3.0:1789760741.633257:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725847980164 time 725897687744 state 0x0 ref 1 00000100:00100000:3.0:1789760741.633270:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725897948160 time 725897687744 state 0x1 ref 1 00000100:00100000:3.0:1789760741.633332:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725897948160 time 725954048540 state 0x0 ref 1 00000100:00100000:3.0:1789760741.633345:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495812480:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760741.633465:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23429120, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:3.0:1789760741.633475:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546434 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.633490:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760741.633495:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.633500:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546434 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.633513:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546434 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.667067:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973037 is committed 00002000:00010000:3.0:1789760741.667337:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23429120 -> 23433216 00002000:00010000:3.0:1789760741.667341:0:6714:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760740 -> 1789760741 00002000:00010000:3.0:1789760741.667344:0:6714:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760740 -> 1789760741 00010000:00010000:3.0:1789760741.667357:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546434 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.667380:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546434 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.667397:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546434 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.667405:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760741.667410:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546434 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.667419:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546434 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.667437:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546434 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.667445:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546434 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.667543:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495812480:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34195us (34540us total) trans 4294973037 rc 0/0 00000100:00100000:3.0:1789760741.667553:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5749 00000100:00100000:3.0:1789760741.667562:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725897948160 time 725954048540 state 0x2 ref 0 00000100:00100000:0.0:1789760741.672181:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760741.672269:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495812608 00000100:00100000:3.0:1789760741.672359:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495812608 00000100:00100000:3.0:1789760741.672411:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725897948160 time 725954048540 state 0x0 ref 1 00000100:00100000:3.0:1789760741.672426:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725954308956 time 725954048540 state 0x1 ref 1 00000100:00100000:3.0:1789760741.672453:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725954308956 time 725993170087 state 0x0 ref 1 00000100:00100000:3.0:1789760741.672466:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04932680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495812608:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760741.672514:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23433216, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:3.0:1789760741.672524:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454643b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.672541:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760741.672545:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.672551:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454643b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.672563:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454643b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760741.718576:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23433216 -> 23437312 00010000:00010000:3.0:1789760741.718595:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454643b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.718609:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454643b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.718618:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454643b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.718626:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760741.718632:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454643b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.718640:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454643b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.718664:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454643b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.718673:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454643b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.718828:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04932680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495812608:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46358us (46649us total) trans 4294973038 rc 0/0 00000100:00100000:3.0:1789760741.718840:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5750 00000100:00100000:3.0:1789760741.718850:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725954308956 time 725993170087 state 0x2 ref 0 00000001:00080000:2.0:1789760741.718983:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973038 is committed 00000100:00100000:1.0:1789760741.726844:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760741.727250:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495812736 00000100:00100000:3.0:1789760741.727260:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495812736 00000100:00100000:3.0:1789760741.727355:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725954308956 time 725993170087 state 0x0 ref 1 00000100:00100000:3.0:1789760741.727371:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725993430503 time 725993170087 state 0x1 ref 1 00000100:00100000:3.0:1789760741.727402:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725993430503 time 726048117801 state 0x0 ref 1 00000100:00100000:3.0:1789760741.727416:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495812736:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760741.727474:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23437312, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:3.0:1789760741.727484:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546442 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.727501:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760741.727506:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.727512:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546442 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.727527:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546442 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760741.764432:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973039 is committed 00002000:00010000:3.0:1789760741.764812:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23437312 -> 23441408 00010000:00010000:3.0:1789760741.764826:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546442 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.764839:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546442 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.764848:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546442 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.764855:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760741.764860:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546442 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.764869:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546442 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.764895:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546442 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.765502:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546442 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.765646:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495812736:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38228us (38806us total) trans 4294973039 rc 0/0 00000100:00100000:3.0:1789760741.765657:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5751 00000100:00100000:3.0:1789760741.765667:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725993430503 time 726048117801 state 0x2 ref 0 00000100:00100000:0.0:1789760741.768933:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760741.769001:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495812864 00000100:00100000:3.0:1789760741.769010:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495812864 00000100:00100000:3.0:1789760741.769171:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 725993430503 time 726048117801 state 0x0 ref 1 00000100:00100000:3.0:1789760741.769184:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726048378217 time 726048117801 state 0x1 ref 1 00000100:00100000:3.0:1789760741.769214:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726048378217 time 726089931241 state 0x0 ref 1 00000100:00100000:3.0:1789760741.769228:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04930a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495812864:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760741.769307:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23441408, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:3.0:1789760741.769316:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546449 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.769331:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760741.769336:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.769341:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546449 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.769354:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546449 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760741.796980:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973040 is committed 00002000:00010000:3.0:1789760741.796993:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23441408 -> 23445504 00010000:00010000:3.0:1789760741.797007:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546449 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.797027:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546449 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.797036:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546449 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.797054:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760741.797059:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546449 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.797069:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546449 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.797093:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546449 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.797101:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546449 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.797199:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04930a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495812864:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27970us (28271us total) trans 4294973040 rc 0/0 00000100:00100000:3.0:1789760741.797210:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5752 00000100:00100000:3.0:1789760741.797220:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726048378217 time 726089931241 state 0x2 ref 0 00000100:00100000:0.0:1789760741.800340:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760741.800492:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495812992 00000100:00100000:3.0:1789760741.800503:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495812992 00000100:00100000:3.0:1789760741.800550:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726048378217 time 726089931241 state 0x0 ref 1 00000100:00100000:3.0:1789760741.800567:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726090191657 time 726089931241 state 0x1 ref 1 00000100:00100000:3.0:1789760741.800585:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726090191657 time 726121302743 state 0x0 ref 1 00000100:00100000:3.0:1789760741.800595:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04931180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495812992:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760741.800636:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23445504, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:3.0:1789760741.800644:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546450 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.800659:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760741.800663:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.800669:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546450 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.800683:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546450 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760741.845721:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973041 is committed 00002000:00010000:3.0:1789760741.845825:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23445504 -> 23449600 00010000:00010000:3.0:1789760741.845841:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546450 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.845854:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546450 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.845863:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546450 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.845871:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760741.845876:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546450 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.845885:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546450 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.845905:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546450 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.845913:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546450 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.846057:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495812992:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45457us (45719us total) trans 4294973041 rc 0/0 00000100:00100000:3.0:1789760741.846070:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5753 00000100:00100000:3.0:1789760741.846080:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726090191657 time 726121302743 state 0x2 ref 0 00000100:00100000:2.0:1789760741.848935:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760741.849016:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495813120 00000100:00100000:3.0:1789760741.849030:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495813120 00000100:00100000:3.0:1789760741.849094:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726090191657 time 726121302743 state 0x0 ref 1 00000100:00100000:3.0:1789760741.849108:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726121563159 time 726121302743 state 0x1 ref 1 00000100:00100000:3.0:1789760741.849139:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726121563159 time 726169856029 state 0x0 ref 1 00000100:00100000:3.0:1789760741.849152:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495813120:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760741.849201:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23449600, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:3.0:1789760741.849210:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546457 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.849226:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760741.849230:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.849237:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546457 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.849251:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546457 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.915399:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973042 is committed 00002000:00010000:3.0:1789760741.915530:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23449600 -> 23453696 00010000:00010000:3.0:1789760741.915550:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546457 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.915566:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546457 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.915575:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546457 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.915583:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760741.915588:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546457 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.915605:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546457 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.915624:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546457 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.915629:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546457 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.915707:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495813120:12345-192.168.203.9@tcp:4:dd.0 Request processed in 66553us (66781us total) trans 4294973042 rc 0/0 00000100:00100000:3.0:1789760741.915717:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5754 00000100:00100000:3.0:1789760741.915727:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726121563159 time 726169856029 state 0x2 ref 0 00000100:00100000:1.0:1789760741.923259:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760741.923609:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495813248 00000100:00100000:3.0:1789760741.923620:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495813248 00000100:00100000:3.0:1789760741.923676:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726121563159 time 726169856029 state 0x0 ref 1 00000100:00100000:3.0:1789760741.923692:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726170116445 time 726169856029 state 0x1 ref 1 00000100:00100000:3.0:1789760741.923723:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726170116445 time 726244440356 state 0x0 ref 1 00000100:00100000:3.0:1789760741.923737:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495813248:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760741.923789:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23453696, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:3.0:1789760741.923803:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454645e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.923820:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760741.923825:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.923832:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454645e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.923847:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454645e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.962799:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973043 is committed 00002000:00010000:2.0:1789760741.962914:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23453696 -> 23457792 00010000:00010000:2.0:1789760741.962932:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454645e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.962944:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454645e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.962953:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454645e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.962960:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760741.962965:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454645e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.962973:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454645e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.962993:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454645e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.963002:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454645e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760741.963175:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495813248:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39436us (39920us total) trans 4294973043 rc 0/0 00000100:00100000:2.0:1789760741.963186:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5755 00000100:00100000:2.0:1789760741.963197:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726170116445 time 726244440356 state 0x2 ref 0 00000100:00100000:0.0:1789760741.966377:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760741.966494:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495813376 00000100:00100000:3.0:1789760741.966505:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495813376 00000100:00100000:3.0:1789760741.966555:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726170116445 time 726244440356 state 0x0 ref 1 00000100:00100000:3.0:1789760741.966571:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726244700772 time 726244440356 state 0x1 ref 1 00000100:00100000:3.0:1789760741.966599:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726244700772 time 726287316298 state 0x0 ref 1 00000100:00100000:3.0:1789760741.966616:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04931f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495813376:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760741.966663:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23457792, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:3.0:1789760741.966672:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546465 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.966690:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760741.966695:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.966702:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546465 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.966716:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546465 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760741.995536:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973044 is committed 00002000:00010000:3.0:1789760741.995644:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23457792 -> 23461888 00010000:00010000:3.0:1789760741.995660:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546465 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.995673:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546465 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.995682:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546465 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.995689:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760741.995693:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546465 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.995703:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546465 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.995722:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546465 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.995729:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546465 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.995818:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495813376:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29198us (29452us total) trans 4294973044 rc 0/0 00000100:00100000:3.0:1789760741.995828:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5756 00000100:00100000:3.0:1789760741.995836:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726244700772 time 726287316298 state 0x2 ref 0 00000100:00100000:3.0:1789760741.999005:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760741.999168:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495813504 00000100:00100000:0.0:1789760741.999178:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495813504 00000100:00100000:0.0:1789760741.999237:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726244700772 time 726287316298 state 0x0 ref 1 00000100:00100000:0.0:1789760741.999251:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726287576714 time 726287316298 state 0x1 ref 1 00000100:00100000:0.0:1789760741.999316:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726287576714 time 726320032207 state 0x0 ref 1 00000100:00100000:0.0:1789760741.999328:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495813504:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760741.999477:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23461888, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:0.0:1789760741.999487:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454646c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.999503:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760741.999507:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.999512:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454646c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.999527:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454646c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760742.033149:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973045 is committed 00002000:00010000:1.0:1789760742.033326:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23461888 -> 23465984 00010000:00010000:1.0:1789760742.033341:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454646c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.033355:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454646c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.033376:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454646c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.033385:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94aa00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760742.033390:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454646c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.033399:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454646c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.033419:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454646c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.033427:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454646c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.033513:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495813504:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34183us (34512us total) trans 4294973045 rc 0/0 00000100:00100000:1.0:1789760742.033523:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5757 00000100:00100000:1.0:1789760742.033534:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726287576714 time 726320032207 state 0x2 ref 0 00000100:00100000:0.0:1789760742.036782:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760742.036858:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495813632 00000100:00100000:2.0:1789760742.036868:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495813632 00000100:00100000:2.0:1789760742.036918:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726287576714 time 726320032207 state 0x0 ref 1 00000100:00100000:2.0:1789760742.036932:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726320292623 time 726320032207 state 0x1 ref 1 00000100:00100000:2.0:1789760742.036961:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726320292623 time 726357678708 state 0x0 ref 1 00000100:00100000:2.0:1789760742.036974:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495813632:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760742.037094:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23465984, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:2.0:1789760742.037103:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546473 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.037117:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760742.037122:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.037128:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546473 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.037147:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546473 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760742.069340:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973046 is committed 00002000:00010000:3.0:1789760742.069501:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23465984 -> 23470080 00010000:00010000:3.0:1789760742.069517:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546473 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.069529:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546473 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.069537:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546473 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.069548:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760742.069552:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546473 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.069560:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546473 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.069578:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546473 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.069585:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546473 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.069713:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495813632:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32736us (32935us total) trans 4294973046 rc 0/0 00000100:00100000:3.0:1789760742.069725:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5758 00000100:00100000:3.0:1789760742.069734:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726320292623 time 726357678708 state 0x2 ref 0 00000100:00100000:0.0:1789760742.073769:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760742.073852:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495813760 00000100:00100000:3.0:1789760742.073861:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495813760 00000100:00100000:3.0:1789760742.073909:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726320292623 time 726357678708 state 0x0 ref 1 00000100:00100000:3.0:1789760742.073924:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726357939124 time 726357678708 state 0x1 ref 1 00000100:00100000:3.0:1789760742.073999:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726357939124 time 726394715806 state 0x0 ref 1 00000100:00100000:3.0:1789760742.074082:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495813760:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760742.074132:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23470080, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:3.0:1789760742.074141:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454647a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.074159:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760742.074163:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.074170:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454647a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.074184:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454647a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.097968:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973047 is committed 00002000:00010000:3.0:1789760742.098120:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23470080 -> 23474176 00010000:00010000:3.0:1789760742.098135:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454647a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.098149:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454647a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.098158:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454647a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.098166:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760742.098170:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454647a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.098180:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454647a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.098198:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454647a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.098205:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454647a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.098370:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495813760:12345-192.168.203.9@tcp:4:dd.0 Request processed in 24343us (24597us total) trans 4294973047 rc 0/0 00000100:00100000:3.0:1789760742.098381:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5759 00000100:00100000:3.0:1789760742.098392:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726357939124 time 726394715806 state 0x2 ref 0 00000100:00100000:2.0:1789760742.101437:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760742.101533:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495813888 00000100:00100000:1.0:1789760742.101542:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495813888 00000100:00100000:1.0:1789760742.101594:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726357939124 time 726394715806 state 0x0 ref 1 00000100:00100000:1.0:1789760742.101610:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726394976222 time 726394715806 state 0x1 ref 1 00000100:00100000:1.0:1789760742.101669:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726394976222 time 726422384647 state 0x0 ref 1 00000100:00100000:1.0:1789760742.101681:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04291180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495813888:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760742.101726:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23474176, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:1.0:1789760742.101736:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546481 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.101752:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760742.101756:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.101762:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546481 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.101774:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546481 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.129974:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973048 is committed 00002000:00010000:0.0:1789760742.130010:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23474176 -> 23478272 00010000:00010000:0.0:1789760742.130173:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546481 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.130186:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546481 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.130195:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546481 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.130204:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760742.130209:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546481 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.130220:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546481 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.130365:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546481 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.130374:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546481 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760742.130469:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04291180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495813888:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28783us (29189us total) trans 4294973048 rc 0/0 00000100:00100000:0.0:1789760742.130535:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5760 00000100:00100000:0.0:1789760742.130546:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726394976222 time 726422384647 state 0x2 ref 0 00000100:00100000:0.0:1789760742.133939:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760742.134326:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495814016 00000100:00100000:3.0:1789760742.134337:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495814016 00000100:00100000:3.0:1789760742.134442:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726394976222 time 726422384647 state 0x0 ref 1 00000100:00100000:3.0:1789760742.134458:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726422645063 time 726422384647 state 0x1 ref 1 00000100:00100000:3.0:1789760742.134537:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726422645063 time 726455253134 state 0x0 ref 1 00000100:00100000:3.0:1789760742.134550:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495814016:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760742.134600:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23478272, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:3.0:1789760742.134610:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546488 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.134627:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760742.134631:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.134638:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546488 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.134651:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546488 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760742.165048:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973049 is committed 00002000:00010000:2.0:1789760742.165193:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23478272 -> 23482368 00010000:00010000:2.0:1789760742.165209:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546488 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.165222:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546488 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.165231:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546488 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.165239:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760742.165244:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546488 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.165252:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546488 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.165270:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546488 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.165295:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546488 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.165485:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495814016:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30931us (31549us total) trans 4294973049 rc 0/0 00000100:00100000:2.0:1789760742.165496:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5761 00000100:00100000:2.0:1789760742.165506:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726422645063 time 726455253134 state 0x2 ref 0 00000100:00100000:1.0:1789760742.168265:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760742.168529:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495814144 00000100:00100000:3.0:1789760742.168538:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495814144 00000100:00100000:3.0:1789760742.168590:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726422645063 time 726455253134 state 0x0 ref 1 00000100:00100000:3.0:1789760742.168607:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726455513550 time 726455253134 state 0x1 ref 1 00000100:00100000:3.0:1789760742.168670:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726455513550 time 726489387421 state 0x0 ref 1 00000100:00100000:3.0:1789760742.168681:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495814144:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760742.168724:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23482368, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:3.0:1789760742.168732:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454648f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.168746:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760742.168749:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.168754:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454648f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.168767:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454648f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760742.202905:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973050 is committed 00002000:00010000:1.0:1789760742.202917:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23482368 -> 23486464 00010000:00010000:1.0:1789760742.202933:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454648f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.202947:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454648f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.202956:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454648f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.202964:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760742.202970:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454648f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.202979:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454648f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.203003:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454648f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.203011:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454648f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.203330:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495814144:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34644us (35068us total) trans 4294973050 rc 0/0 00000100:00100000:1.0:1789760742.203343:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5762 00000100:00100000:1.0:1789760742.203354:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726455513550 time 726489387421 state 0x2 ref 0 00000100:00100000:0.0:1789760742.210361:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760742.210471:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495814272 00000100:00100000:1.0:1789760742.210481:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495814272 00000100:00100000:1.0:1789760742.210539:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726455513550 time 726489387421 state 0x0 ref 1 00000100:00100000:1.0:1789760742.210555:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726489647837 time 726489387421 state 0x1 ref 1 00000100:00100000:1.0:1789760742.210586:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726489647837 time 726531303493 state 0x0 ref 1 00000100:00100000:1.0:1789760742.210598:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495814272:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760742.210646:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23486464, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:1.0:1789760742.210654:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546496 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.210671:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760742.210675:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.210682:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546496 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.210697:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546496 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.238706:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760742.259499:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973051 is committed 00002000:00010000:1.0:1789760742.259642:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23486464 -> 23490560 00010000:00010000:1.0:1789760742.259657:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546496 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.259670:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546496 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.259686:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546496 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.259693:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760742.259698:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546496 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.259707:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546496 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.259725:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546496 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.259734:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546496 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.259821:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495814272:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49220us (49464us total) trans 4294973051 rc 0/0 00000100:00100000:1.0:1789760742.259831:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5763 00000100:00100000:1.0:1789760742.259840:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726489647837 time 726531303493 state 0x2 ref 0 00000100:00100000:0.0:1789760742.263043:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760742.263249:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495814400 00000100:00100000:2.0:1789760742.263263:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495814400 00000100:00100000:2.0:1789760742.263424:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726489647837 time 726531303493 state 0x0 ref 1 00000100:00100000:2.0:1789760742.263438:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726531563909 time 726531303493 state 0x1 ref 1 00000100:00100000:2.0:1789760742.263498:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726531563909 time 726584215050 state 0x0 ref 1 00000100:00100000:2.0:1789760742.263510:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495814400:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760742.263618:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23490560, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:2.0:1789760742.263627:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454649d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.263643:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760742.263648:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.263654:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454649d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.263668:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454649d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.297086:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973052 is committed 00002000:00010000:3.0:1789760742.297203:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23490560 -> 23494656 00010000:00010000:3.0:1789760742.297220:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454649d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.297232:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454649d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.297242:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454649d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.297249:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760742.297253:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454649d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.297261:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454649d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.297405:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454649d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.297414:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454649d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760742.297567:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495814400:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34054us (34545us total) trans 4294973052 rc 0/0 00000100:00100000:0.0:1789760742.297579:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5764 00000100:00100000:0.0:1789760742.297589:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726531563909 time 726584215050 state 0x2 ref 0 00000100:00100000:2.0:1789760742.301488:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760742.301667:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495814528 00000100:00100000:1.0:1789760742.301678:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495814528 00000100:00100000:1.0:1789760742.301734:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726531563909 time 726584215050 state 0x0 ref 1 00000100:00100000:1.0:1789760742.301754:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726584475466 time 726584215050 state 0x1 ref 1 00000100:00100000:1.0:1789760742.301816:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726584475466 time 726622532094 state 0x0 ref 1 00000100:00100000:1.0:1789760742.301830:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495814528:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760742.301880:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23494656, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:1.0:1789760742.301890:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464a4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.301906:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760742.301911:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.301918:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.301931:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760742.328555:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973053 is committed 00002000:00010000:1.0:1789760742.328685:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23494656 -> 23498752 00010000:00010000:1.0:1789760742.328700:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.328713:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464a4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.328722:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.328735:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760742.328739:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.328748:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464a4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.328768:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464a4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.328776:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464a4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.329000:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495814528:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27166us (27515us total) trans 4294973053 rc 0/0 00000100:00100000:1.0:1789760742.329012:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5765 00000100:00100000:1.0:1789760742.329032:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726584475466 time 726622532094 state 0x2 ref 0 00000100:00100000:1.0:1789760742.333501:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760742.333658:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495814656 00000100:00100000:0.0:1789760742.333669:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495814656 00000100:00100000:0.0:1789760742.333722:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726584475466 time 726622532094 state 0x0 ref 1 00000100:00100000:0.0:1789760742.333737:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726622792510 time 726622532094 state 0x1 ref 1 00000100:00100000:0.0:1789760742.333805:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726622792510 time 726654520850 state 0x0 ref 1 00000100:00100000:0.0:1789760742.333819:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495814656:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760742.333869:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23498752, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:0.0:1789760742.333878:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545464ab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.333895:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760742.333899:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.333905:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545464ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.333924:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545464ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760742.362895:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973054 is committed 00002000:00010000:0.0:1789760742.363265:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23498752 -> 23502848 00010000:00010000:0.0:1789760742.363296:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545464ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.363309:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545464ab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.363320:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545464ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.363327:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760742.363333:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545464ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.363342:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545464ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.363426:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545464ab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.363434:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545464ab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760742.363567:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495814656:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29746us (30070us total) trans 4294973054 rc 0/0 00000100:00100000:0.0:1789760742.363577:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5766 00000100:00100000:0.0:1789760742.363587:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726622792510 time 726654520850 state 0x2 ref 0 00000100:00100000:2.0:1789760742.368506:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760742.369220:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495814784 00000100:00100000:0.0:1789760742.369228:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495814784 00000100:00100000:0.0:1789760742.369302:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726622792510 time 726654520850 state 0x0 ref 1 00000100:00100000:0.0:1789760742.369318:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726654781266 time 726654520850 state 0x1 ref 1 00000100:00100000:0.0:1789760742.369342:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726654781266 time 726690059722 state 0x0 ref 1 00000100:00100000:0.0:1789760742.369354:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04292a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495814784:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760742.369712:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23502848, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:0.0:1789760742.369720:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545464b2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.369737:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760742.369741:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.369747:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545464b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.369761:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545464b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760742.393635:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973055 is committed 00002000:00010000:3.0:1789760742.393810:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23502848 -> 23506944 00010000:00010000:3.0:1789760742.393828:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545464b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.393842:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545464b2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.393852:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545464b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.393861:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760742.393866:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545464b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.393875:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545464b2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.393899:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545464b2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.393908:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545464b2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.394004:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04292a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495814784:12345-192.168.203.9@tcp:4:dd.0 Request processed in 24646us (25502us total) trans 4294973055 rc 0/0 00000100:00100000:3.0:1789760742.394094:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5767 00000100:00100000:3.0:1789760742.394104:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726654781266 time 726690059722 state 0x2 ref 0 00000100:00100000:0.0:1789760742.397615:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760742.397714:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495814912 00000100:00100000:1.0:1789760742.397724:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495814912 00000100:00100000:1.0:1789760742.397783:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726654781266 time 726690059722 state 0x0 ref 1 00000100:00100000:1.0:1789760742.397796:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726690320138 time 726690059722 state 0x1 ref 1 00000100:00100000:1.0:1789760742.397824:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726690320138 time 726718540734 state 0x0 ref 1 00000100:00100000:1.0:1789760742.397837:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495814912:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760742.397883:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23506944, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:1.0:1789760742.397892:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464b9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.397908:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760742.397913:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.397919:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.397933:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.441745:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973056 is committed 00002000:00010000:1.0:1789760742.441809:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23506944 -> 23511040 00010000:00010000:1.0:1789760742.441823:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.441836:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464b9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.441846:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.441853:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760742.441858:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.441868:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464b9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.441887:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464b9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.441895:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464b9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.441976:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495814912:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44136us (44367us total) trans 4294973056 rc 0/0 00000100:00100000:1.0:1789760742.441987:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5768 00000100:00100000:1.0:1789760742.441997:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726690320138 time 726718540734 state 0x2 ref 0 00000100:00100000:3.0:1789760742.444883:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760742.444994:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495815040 00000100:00100000:1.0:1789760742.445004:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495815040 00000100:00100000:1.0:1789760742.445075:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726690320138 time 726718540734 state 0x0 ref 1 00000100:00100000:1.0:1789760742.445090:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726718801150 time 726718540734 state 0x1 ref 1 00000100:00100000:1.0:1789760742.445119:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726718801150 time 726765836130 state 0x0 ref 1 00000100:00100000:1.0:1789760742.445131:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495815040:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760742.445184:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23511040, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:1.0:1789760742.445194:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545464c0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.445211:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760742.445216:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.445223:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545464c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.445238:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545464c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760742.468183:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973057 is committed 00002000:00010000:1.0:1789760742.468257:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23511040 -> 23515136 00010000:00010000:1.0:1789760742.468271:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545464c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.468303:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545464c0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.468313:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545464c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.468320:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760742.468326:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545464c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.468336:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545464c0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.468420:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545464c0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.468431:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545464c0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.468545:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495815040:12345-192.168.203.9@tcp:4:dd.0 Request processed in 23411us (23666us total) trans 4294973057 rc 0/0 00000100:00100000:1.0:1789760742.468555:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5769 00000100:00100000:1.0:1789760742.468564:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726718801150 time 726765836130 state 0x2 ref 0 00000100:00100000:1.0:1789760742.471794:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760742.471863:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495815168 00000100:00100000:3.0:1789760742.471872:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495815168 00000100:00100000:3.0:1789760742.471919:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726718801150 time 726765836130 state 0x0 ref 1 00000100:00100000:3.0:1789760742.471934:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726766096546 time 726765836130 state 0x1 ref 1 00000100:00100000:3.0:1789760742.471963:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726766096546 time 726792680795 state 0x0 ref 1 00000100:00100000:3.0:1789760742.471975:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495815168:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760742.472028:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23515136, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:3.0:1789760742.472045:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545464c7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.472061:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760742.472066:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.472072:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545464c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.472086:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545464c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760742.493815:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973058 is committed 00002000:00010000:3.0:1789760742.493928:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23515136 -> 23519232 00010000:00010000:3.0:1789760742.493942:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545464c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.493956:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545464c7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.493966:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545464c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.493974:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760742.493979:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545464c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.493988:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545464c7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.494009:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545464c7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.494027:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545464c7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.494124:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495815168:12345-192.168.203.9@tcp:4:dd.0 Request processed in 22146us (22333us total) trans 4294973058 rc 0/0 00000100:00100000:3.0:1789760742.494135:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5770 00000100:00100000:3.0:1789760742.494147:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726766096546 time 726792680795 state 0x2 ref 0 00000100:00100000:0.0:1789760742.497253:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760742.497482:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495815296 00000100:00100000:1.0:1789760742.497492:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495815296 00000100:00100000:1.0:1789760742.497543:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726766096546 time 726792680795 state 0x0 ref 1 00000100:00100000:1.0:1789760742.497557:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726792941211 time 726792680795 state 0x1 ref 1 00000100:00100000:1.0:1789760742.497616:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726792941211 time 726818332535 state 0x0 ref 1 00000100:00100000:1.0:1789760742.497628:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495815296:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760742.497678:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23519232, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:1.0:1789760742.497687:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545464ce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.497703:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760742.497707:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.497713:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545464ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.497728:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545464ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760742.519059:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973059 is committed 00002000:00010000:1.0:1789760742.519215:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23519232 -> 23523328 00010000:00010000:1.0:1789760742.519230:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545464ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.519243:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545464ce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.519251:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545464ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.519258:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760742.519263:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545464ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.519272:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545464ce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.519312:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545464ce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.519319:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545464ce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.519495:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495815296:12345-192.168.203.9@tcp:4:dd.0 Request processed in 21862us (22243us total) trans 4294973059 rc 0/0 00000100:00100000:1.0:1789760742.519505:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5771 00000100:00100000:1.0:1789760742.519515:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726792941211 time 726818332535 state 0x2 ref 0 00000100:00100000:2.0:1789760742.524199:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760742.524323:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495815424 00000100:00100000:1.0:1789760742.524332:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495815424 00000100:00100000:1.0:1789760742.524397:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726792941211 time 726818332535 state 0x0 ref 1 00000100:00100000:1.0:1789760742.524412:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726818592951 time 726818332535 state 0x1 ref 1 00000100:00100000:1.0:1789760742.524435:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726818592951 time 726845151962 state 0x0 ref 1 00000100:00100000:1.0:1789760742.524449:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f921c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495815424:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760742.524493:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23523328, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:1.0:1789760742.524502:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464d5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.524518:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760742.524522:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.524529:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.524543:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760742.556841:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973060 is committed 00002000:00010000:1.0:1789760742.556957:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23523328 -> 23527424 00010000:00010000:1.0:1789760742.556974:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.556987:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464d5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.556995:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.557003:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760742.557008:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.557030:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464d5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.557058:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464d5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.557066:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464d5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.557156:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f921c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495815424:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32706us (32963us total) trans 4294973060 rc 0/0 00000100:00100000:1.0:1789760742.557170:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5772 00000100:00100000:1.0:1789760742.557180:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726818592951 time 726845151962 state 0x2 ref 0 00000100:00100000:2.0:1789760742.560679:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760742.560754:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495815552 00000100:00100000:3.0:1789760742.560764:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495815552 00000100:00100000:3.0:1789760742.560813:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726818592951 time 726845151962 state 0x0 ref 1 00000100:00100000:3.0:1789760742.560826:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726845412378 time 726845151962 state 0x1 ref 1 00000100:00100000:3.0:1789760742.560879:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726845412378 time 726881595798 state 0x0 ref 1 00000100:00100000:3.0:1789760742.560891:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f923100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495815552:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760742.560935:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23527424, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:3.0:1789760742.560945:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545464dc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.560960:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760742.560964:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.560971:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545464dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.560984:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545464dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760742.583933:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973061 is committed 00002000:00010000:2.0:1789760742.584049:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23527424 -> 23531520 00010000:00010000:2.0:1789760742.584071:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545464dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.584085:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545464dc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.584094:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545464dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.584102:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760742.584107:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545464dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.584117:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545464dc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.584135:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545464dc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.584148:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545464dc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.584237:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f923100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495815552:12345-192.168.203.9@tcp:4:dd.0 Request processed in 23343us (23562us total) trans 4294973061 rc 0/0 00000100:00100000:2.0:1789760742.584246:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5773 00000100:00100000:2.0:1789760742.584256:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726845412378 time 726881595798 state 0x2 ref 0 00000100:00100000:0.0:1789760742.588553:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760742.588662:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495815680 00000100:00100000:2.0:1789760742.588672:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495815680 00000100:00100000:2.0:1789760742.588726:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726845412378 time 726881595798 state 0x0 ref 1 00000100:00100000:2.0:1789760742.588740:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726881856214 time 726881595798 state 0x1 ref 1 00000100:00100000:2.0:1789760742.588763:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726881856214 time 726909479938 state 0x0 ref 1 00000100:00100000:2.0:1789760742.588775:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495815680:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760742.588819:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23531520, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:2.0:1789760742.588829:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545464e3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.588845:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760742.588850:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.588856:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545464e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.588875:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545464e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760742.623222:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23531520 -> 23535616 00010000:00010000:0.0:1789760742.623238:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545464e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.623248:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545464e3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.623254:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545464e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.623258:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760742.623262:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545464e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.623268:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545464e3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.623305:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545464e3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.623318:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545464e3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.623338:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973062 is committed 00000100:00100000:0.0:1789760742.623489:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495815680:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34710us (34938us total) trans 4294973062 rc 0/0 00000100:00100000:0.0:1789760742.623500:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5774 00000100:00100000:0.0:1789760742.623510:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726881856214 time 726909479938 state 0x2 ref 0 00000100:00100000:0.0:1789760742.626149:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760742.626276:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495815808 00000100:00100000:1.0:1789760742.626471:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495815808 00000100:00100000:1.0:1789760742.627042:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726881856214 time 726909479938 state 0x0 ref 1 00000100:00100000:1.0:1789760742.627057:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726909740354 time 726909479938 state 0x1 ref 1 00000100:00100000:1.0:1789760742.627103:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726909740354 time 726947819398 state 0x0 ref 1 00000100:00100000:1.0:1789760742.627116:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495815808:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760742.627161:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23535616, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:1.0:1789760742.627171:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464ea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.627187:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760742.627192:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.627198:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.627211:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760742.663181:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973063 is committed 00002000:00010000:1.0:1789760742.663400:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23535616 -> 23539712 00002000:00010000:1.0:1789760742.663405:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760741 -> 1789760742 00002000:00010000:1.0:1789760742.663407:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760741 -> 1789760742 00010000:00010000:1.0:1789760742.663420:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.663432:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464ea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.663440:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.663446:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760742.663451:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.663459:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464ea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.663488:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464ea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.663496:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464ea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.663594:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495815808:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36466us (37442us total) trans 4294973063 rc 0/0 00000100:00100000:1.0:1789760742.663603:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5775 00000100:00100000:1.0:1789760742.663613:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726909740354 time 726947819398 state 0x2 ref 0 00000100:00100000:2.0:1789760742.672393:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760742.672664:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495815936 00000100:00100000:3.0:1789760742.672673:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495815936 00000100:00100000:3.0:1789760742.672719:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726909740354 time 726947819398 state 0x0 ref 1 00000100:00100000:3.0:1789760742.672731:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726948079814 time 726947819398 state 0x1 ref 1 00000100:00100000:3.0:1789760742.672755:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726948079814 time 726993473893 state 0x0 ref 1 00000100:00100000:3.0:1789760742.672765:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f922300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495815936:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760742.672813:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23539712, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:3.0:1789760742.672822:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545464f1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.672837:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760742.672841:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.672848:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545464f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.672861:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545464f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760742.687411:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520307328:0@lo:400:kworker.0 00000100:00100000:2.0:1789760742.687450:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033ce300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:59c4e264-9b75-482b-8813-3d1dc124f384:3317:1876699520307072:0@lo:400:kworker.0 00000100:00100000:2.0:1789760742.687513:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760742.687556:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033cc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3317:1876699520307584:0@lo:400:kworker.0 00000100:00100000:2.0:1789760742.687584:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760742.687608:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1789760742.687614:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033cdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3317:1876699520307200:0@lo:400:kworker.0 00000100:00100000:2.0:1789760742.687635:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760742.687652:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033cea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3317:1876699520307712:0@lo:400:kworker.0 00000100:00100000:2.0:1789760742.687674:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760742.687706:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520307584 00000100:00100000:2.0:1789760742.687714:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520307584 00000100:00100000:2.0:1789760742.687757:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520307200 00000100:00100000:2.0:1789760742.687760:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520307200 00000100:00100000:2.0:1789760742.687782:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 482 00000100:00100000:2.0:1789760742.687790:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1876699520307584:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760742.687838:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1876699520307584:12345-0@lo:400:kworker.0 Request processed in 49us (254us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760742.687845:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 482 00000100:00100000:0.0:1789760742.687855:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760742.687862:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520307328 00000100:00100000:2.0:1789760742.687865:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520307328 00000100:00100000:0.0:1789760742.687874:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789760742.687879:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033ced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520307456:0@lo:400:kworker.0 00000100:00100000:2.0:1789760742.687880:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 483 00000100:00100000:2.0:1789760742.687886:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f89f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1876699520307200:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760742.687902:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760742.687910:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f89f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1876699520307200:12345-0@lo:400:kworker.0 Request processed in 24us (274us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760742.687917:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 483 00000100:00100000:2.0:1789760742.687927:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 484 00000100:00100000:2.0:1789760742.687933:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370bd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520307328:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760742.687939:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520307712 00000100:00100000:0.0:1789760742.687942:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520307712 00000100:00100000:2.0:1789760742.687962:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370bd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520307328:12345-0@lo:400:kworker.0 Request processed in 30us (107us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760742.687965:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520307456 00000100:00100000:0.0:1789760742.687967:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520307456 00000100:00100000:2.0:1789760742.687968:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 484 00000100:00100000:0.0:1789760742.687987:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 224 00000100:00100000:0.0:1789760742.687993:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3317:x1876699520307712:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760742.687999:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033cdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3317:1876699520307200:0@lo:400:kworker.0 00000100:00100000:2.0:1789760742.688009:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033cc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3317:1876699520307584:0@lo:400:kworker.0 00000100:00100000:0.0:1789760742.688021:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3317:x1876699520307712:12345-0@lo:400:kworker.0 Request processed in 29us (346us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760742.688027:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 224 00000100:00100000:0.0:1789760742.688048:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 225 00000100:00100000:0.0:1789760742.688055:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520307456:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760742.688082:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520307456:12345-0@lo:400:kworker.0 Request processed in 28us (179us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760742.688088:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 225 00000100:00100000:0.0:1789760742.688117:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033ced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520307456:0@lo:400:kworker.0 00000100:00100000:0.0:1789760742.688126:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520307328:0@lo:400:kworker.0 00000100:00100000:1.0:1789760742.688141:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520307072 00000100:00100000:1.0:1789760742.688146:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520307072 00000100:00100000:1.0:1789760742.688179:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 401 00000100:00100000:1.0:1789760742.688187:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929012311450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:59c4e264-9b75-482b-8813-3d1dc124f384+19:3317:x1876699520307072:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760742.688222:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929012311450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:59c4e264-9b75-482b-8813-3d1dc124f384+19:3317:x1876699520307072:12345-0@lo:400:kworker.0 Request processed in 37us (709us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760742.688229:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 401 00000100:00080000:1.0:1789760742.688260:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789760742.688264:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789760742.688267:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760742.688439:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033cea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3317:1876699520307712:0@lo:400:kworker.0 00000100:00100000:2.0:1789760742.688449:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033ce300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:59c4e264-9b75-482b-8813-3d1dc124f384:3317:1876699520307072:0@lo:400:kworker.0 00000001:00080000:1.0:1789760742.710087:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973064 is committed 00002000:00010000:2.0:1789760742.710254:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23539712 -> 23543808 00010000:00010000:2.0:1789760742.710269:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545464f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.710350:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545464f1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.710360:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545464f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.710368:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760742.710374:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545464f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.710384:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545464f1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.710405:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545464f1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.710414:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545464f1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.710612:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f922300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495815936:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37843us (38221us total) trans 4294973064 rc 0/0 00000100:00100000:2.0:1789760742.710623:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5776 00000100:00100000:2.0:1789760742.710633:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726948079814 time 726993473893 state 0x2 ref 0 00000100:00100000:3.0:1789760742.717538:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760742.717638:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495816064 00000100:00100000:0.0:1789760742.717648:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495816064 00000100:00100000:0.0:1789760742.717705:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726948079814 time 726993473893 state 0x0 ref 1 00000100:00100000:0.0:1789760742.717723:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726993734309 time 726993473893 state 0x1 ref 1 00000100:00100000:0.0:1789760742.717756:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726993734309 time 727038472584 state 0x0 ref 1 00000100:00100000:0.0:1789760742.717770:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495816064:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760742.717817:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23543808, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:0.0:1789760742.717827:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545464f8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.717844:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760742.717849:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.717856:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545464f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.717869:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545464f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760742.771626:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23543808 -> 23547904 00010000:00010000:3.0:1789760742.771646:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545464f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.771659:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545464f8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.771669:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545464f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.771683:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760742.771688:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545464f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.771698:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545464f8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.771720:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545464f8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.771728:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545464f8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.771825:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495816064:12345-192.168.203.9@tcp:4:dd.0 Request processed in 54053us (54290us total) trans 4294973065 rc 0/0 00000100:00100000:3.0:1789760742.771836:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5777 00000100:00100000:3.0:1789760742.771847:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726993734309 time 727038472584 state 0x2 ref 0 00000001:00080000:1.0:1789760742.772393:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973065 is committed 00000100:00100000:3.0:1789760742.776662:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760742.776815:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495816192 00000100:00100000:1.0:1789760742.776825:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495816192 00000100:00100000:1.0:1789760742.776882:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 726993734309 time 727038472584 state 0x0 ref 1 00000100:00100000:1.0:1789760742.776899:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727038733000 time 727038472584 state 0x1 ref 1 00000100:00100000:1.0:1789760742.776938:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727038733000 time 727097654331 state 0x0 ref 1 00000100:00100000:1.0:1789760742.776951:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495816192:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760742.776999:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23547904, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:1.0:1789760742.777010:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464ff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.777027:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760742.777045:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.777052:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.777067:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760742.803969:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23547904 -> 23552000 00010000:00010000:1.0:1789760742.803990:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.804003:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464ff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.804024:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.804040:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760742.804045:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.804054:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464ff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.804074:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464ff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.804081:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545464ff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.804185:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495816192:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27232us (27528us total) trans 4294973066 rc 0/0 00000100:00100000:1.0:1789760742.804196:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5778 00000100:00100000:1.0:1789760742.804206:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727038733000 time 727097654331 state 0x2 ref 0 00000001:00080000:3.0:1789760742.804896:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973066 is committed 00000100:00100000:2.0:1789760742.812449:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760742.812612:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495816320 00000100:00100000:3.0:1789760742.812622:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495816320 00000100:00100000:3.0:1789760742.812675:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727038733000 time 727097654331 state 0x0 ref 1 00000100:00100000:3.0:1789760742.812690:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727097914747 time 727097654331 state 0x1 ref 1 00000100:00100000:3.0:1789760742.812732:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727097914747 time 727133448649 state 0x0 ref 1 00000100:00100000:3.0:1789760742.812746:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495816320:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760742.812793:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23552000, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:3.0:1789760742.812802:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546506 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.812817:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760742.812822:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.812828:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546506 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.812841:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546506 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.855781:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973067 is committed 00002000:00010000:2.0:1789760742.855886:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23552000 -> 23556096 00010000:00010000:2.0:1789760742.855902:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546506 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.855915:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546506 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.855926:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546506 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.855938:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760742.855944:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546506 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.855953:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546506 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.855971:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546506 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.855979:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546506 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.856088:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495816320:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43340us (43644us total) trans 4294973067 rc 0/0 00000100:00100000:2.0:1789760742.856099:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5779 00000100:00100000:2.0:1789760742.856108:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727097914747 time 727133448649 state 0x2 ref 0 00000100:00100000:2.0:1789760742.859458:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760742.859573:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495816448 00000100:00100000:3.0:1789760742.859582:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495816448 00000100:00100000:3.0:1789760742.859635:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727097914747 time 727133448649 state 0x0 ref 1 00000100:00100000:3.0:1789760742.859650:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727133709065 time 727133448649 state 0x1 ref 1 00000100:00100000:3.0:1789760742.859681:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727133709065 time 727180397940 state 0x0 ref 1 00000100:00100000:3.0:1789760742.859693:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495816448:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760742.859740:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23556096, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:3.0:1789760742.859749:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454650d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.859765:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760742.859769:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.859774:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454650d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.859786:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454650d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760742.887717:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23556096 -> 23560192 00010000:00010000:3.0:1789760742.887737:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454650d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.887752:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454650d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.887761:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454650d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.887768:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760742.887773:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454650d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.887783:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454650d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.887808:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454650d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.887816:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454650d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.887912:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495816448:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28215us (28457us total) trans 4294973068 rc 0/0 00000100:00100000:3.0:1789760742.887923:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5780 00000100:00100000:3.0:1789760742.887934:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727133709065 time 727180397940 state 0x2 ref 0 00000001:00080000:1.0:1789760742.888007:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973068 is committed 00000100:00100000:3.0:1789760742.891552:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760742.891838:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495816576 00000100:00100000:1.0:1789760742.891848:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495816576 00000100:00100000:1.0:1789760742.891904:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727133709065 time 727180397940 state 0x0 ref 1 00000100:00100000:1.0:1789760742.891920:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727180658356 time 727180397940 state 0x1 ref 1 00000100:00100000:1.0:1789760742.891950:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727180658356 time 727212666504 state 0x0 ref 1 00000100:00100000:1.0:1789760742.891962:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495816576:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760742.892005:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23560192, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:1.0:1789760742.892026:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546514 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.892049:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760742.892054:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.892061:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546514 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.892081:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546514 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760742.920891:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23560192 -> 23564288 00010000:00010000:1.0:1789760742.920911:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546514 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.920924:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546514 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.920934:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546514 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.920941:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760742.920946:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546514 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.920955:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546514 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.920974:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546514 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.920983:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546514 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.921101:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495816576:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29136us (29553us total) trans 4294973069 rc 0/0 00000100:00100000:1.0:1789760742.921113:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5781 00000100:00100000:1.0:1789760742.921123:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727180658356 time 727212666504 state 0x2 ref 0 00000001:00080000:3.0:1789760742.921908:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973069 is committed 00000100:00100000:2.0:1789760742.924988:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760742.925106:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495816704 00000100:00100000:3.0:1789760742.925117:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495816704 00000100:00100000:3.0:1789760742.925170:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727180658356 time 727212666504 state 0x0 ref 1 00000100:00100000:3.0:1789760742.925186:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727212926920 time 727212666504 state 0x1 ref 1 00000100:00100000:3.0:1789760742.925221:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727212926920 time 727245937810 state 0x0 ref 1 00000100:00100000:3.0:1789760742.925234:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903639fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495816704:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760742.925464:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23564288, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:3.0:1789760742.925476:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454651b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.925503:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760742.925603:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.925613:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454651b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.925631:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454651b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760742.964901:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973070 is committed 00002000:00010000:2.0:1789760742.965205:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23564288 -> 23568384 00010000:00010000:2.0:1789760742.965221:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454651b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.965234:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454651b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.965244:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454651b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.965252:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760742.965257:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454651b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.965267:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454651b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.965606:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454651b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.965617:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454651b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.965711:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903639fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495816704:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40468us (40721us total) trans 4294973070 rc 0/0 00000100:00100000:2.0:1789760742.965723:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5782 00000100:00100000:2.0:1789760742.965733:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727212926920 time 727245937810 state 0x2 ref 0 00000100:00100000:2.0:1789760742.972981:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760742.973135:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495816832 00000100:00100000:3.0:1789760742.973143:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495816832 00000100:00100000:3.0:1789760742.973197:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727212926920 time 727245937810 state 0x0 ref 1 00000100:00100000:3.0:1789760742.973211:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727246198226 time 727245937810 state 0x1 ref 1 00000100:00100000:3.0:1789760742.973374:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727246198226 time 727294089511 state 0x0 ref 1 00000100:00100000:3.0:1789760742.973386:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495816832:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760742.973431:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23568384, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:3.0:1789760742.973440:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546522 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.973457:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760742.973461:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.973467:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546522 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.973482:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546522 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.999774:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973071 is committed 00002000:00010000:3.0:1789760742.999873:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23568384 -> 23572480 00010000:00010000:3.0:1789760742.999888:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546522 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.999901:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546522 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.999912:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546522 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.999919:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760742.999925:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546522 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.999934:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546522 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.999956:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546522 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.999965:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546522 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.000082:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495816832:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26692us (27106us total) trans 4294973071 rc 0/0 00000100:00100000:3.0:1789760743.000092:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5783 00000100:00100000:3.0:1789760743.000103:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727246198226 time 727294089511 state 0x2 ref 0 00000100:00100000:3.0:1789760743.003973:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760743.004536:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495816960 00000100:00100000:1.0:1789760743.004546:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495816960 00000100:00100000:1.0:1789760743.004656:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727246198226 time 727294089511 state 0x0 ref 1 00000100:00100000:1.0:1789760743.004672:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727294349927 time 727294089511 state 0x1 ref 1 00000100:00100000:1.0:1789760743.004704:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727294349927 time 727325421041 state 0x0 ref 1 00000100:00100000:1.0:1789760743.004718:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495816960:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760743.004764:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23572480, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:1.0:1789760743.004774:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546529 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.004789:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760743.004794:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760743.004800:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546529 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.004814:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546529 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760743.009812:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760743.033315:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973072 is committed 00002000:00010000:2.0:1789760743.033483:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23572480 -> 23576576 00010000:00010000:2.0:1789760743.033499:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546529 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.033513:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546529 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.033523:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546529 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.033532:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760743.033536:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546529 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.033546:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546529 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.033565:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546529 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.033574:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546529 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.033669:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495816960:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28948us (29699us total) trans 4294973072 rc 0/0 00000100:00100000:2.0:1789760743.033680:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5784 00000100:00100000:2.0:1789760743.033689:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727294349927 time 727325421041 state 0x2 ref 0 00000100:00100000:3.0:1789760743.036914:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760743.036995:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495817088 00000100:00100000:0.0:1789760743.037015:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495817088 00000100:00100000:0.0:1789760743.037076:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727294349927 time 727325421041 state 0x0 ref 1 00000100:00100000:0.0:1789760743.037091:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727325681457 time 727325421041 state 0x1 ref 1 00000100:00100000:0.0:1789760743.037115:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727325681457 time 727357831689 state 0x0 ref 1 00000100:00100000:0.0:1789760743.037127:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495817088:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760743.037173:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23576576, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:0.0:1789760743.037181:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546530 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.037197:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760743.037202:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760743.037209:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546530 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.037223:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546530 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760743.064164:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23576576 -> 23580672 00010000:00010000:0.0:1789760743.064186:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546530 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.064206:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546530 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.064216:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546530 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.064223:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760743.064228:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546530 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.064237:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546530 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.064258:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546530 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.064266:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546530 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.064399:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495817088:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27269us (27489us total) trans 4294973073 rc 0/0 00000100:00100000:0.0:1789760743.064411:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5785 00000100:00100000:0.0:1789760743.064420:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727325681457 time 727357831689 state 0x2 ref 0 00000001:00080000:2.0:1789760743.066133:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973073 is committed 00000100:00100000:2.0:1789760743.069113:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760743.069234:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495817216 00000100:00100000:3.0:1789760743.069244:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495817216 00000100:00100000:3.0:1789760743.069553:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727325681457 time 727357831689 state 0x0 ref 1 00000100:00100000:3.0:1789760743.069568:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727358092105 time 727357831689 state 0x1 ref 1 00000100:00100000:3.0:1789760743.069600:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727358092105 time 727390316766 state 0x0 ref 1 00000100:00100000:3.0:1789760743.069613:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495817216:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760743.069658:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23580672, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:3.0:1789760743.069667:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546537 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.069683:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760743.069687:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.069694:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546537 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.069714:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546537 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760743.101096:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23580672 -> 23584768 00010000:00010000:2.0:1789760743.101114:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546537 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.101127:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546537 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.101135:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546537 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.101142:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760743.101146:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546537 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.101155:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546537 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.101174:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546537 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.101181:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546537 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.101276:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495817216:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31661us (32167us total) trans 4294973074 rc 0/0 00000100:00100000:2.0:1789760743.101333:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5786 00000100:00100000:2.0:1789760743.101343:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727358092105 time 727390316766 state 0x2 ref 0 00000001:00080000:2.0:1789760743.101383:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973074 is committed 00000100:00100000:1.0:1789760743.106661:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760743.109398:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495817344 00000100:00100000:2.0:1789760743.109412:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495817344 00000100:00100000:2.0:1789760743.109470:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727358092105 time 727390316766 state 0x0 ref 1 00000100:00100000:2.0:1789760743.109484:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727390577182 time 727390316766 state 0x1 ref 1 00000100:00100000:2.0:1789760743.109510:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727390577182 time 727430227048 state 0x0 ref 1 00000100:00100000:2.0:1789760743.109834:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495817344:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760743.109881:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23584768, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:2.0:1789760743.109890:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454653e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.109906:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760743.109910:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.109916:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454653e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.109931:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454653e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760743.142438:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973075 is committed 00002000:00010000:2.0:1789760743.142573:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23584768 -> 23588864 00010000:00010000:2.0:1789760743.142590:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454653e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.142608:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454653e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.142618:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454653e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.142625:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760743.142629:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454653e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.142638:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454653e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.142658:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454653e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.142665:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454653e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.142793:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495817344:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32958us (36135us total) trans 4294973075 rc 0/0 00000100:00100000:2.0:1789760743.142805:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5787 00000100:00100000:2.0:1789760743.142816:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727390577182 time 727430227048 state 0x2 ref 0 00000100:00100000:0.0:1789760743.152385:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760743.152611:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495817472 00000100:00100000:2.0:1789760743.152621:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495817472 00000100:00100000:1.0:1789760743.152851:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727390577182 time 727430227048 state 0x0 ref 1 00000100:00100000:1.0:1789760743.152867:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727430487464 time 727430227048 state 0x1 ref 1 00000100:00100000:1.0:1789760743.152940:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727430487464 time 727473655863 state 0x0 ref 1 00000100:00100000:1.0:1789760743.152952:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495817472:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760743.153000:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23588864, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:1.0:1789760743.153018:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546545 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.153042:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760743.153047:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760743.153053:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546545 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.153067:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546545 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760743.178965:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23588864 -> 23592960 00010000:00010000:1.0:1789760743.178983:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546545 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.178996:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546545 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.179005:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546545 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.179023:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760743.179036:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546545 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.179044:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546545 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.179063:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546545 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.179070:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546545 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760743.179166:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495817472:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26212us (26786us total) trans 4294973076 rc 0/0 00000100:00100000:1.0:1789760743.179176:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5788 00000100:00100000:1.0:1789760743.179187:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727430487464 time 727473655863 state 0x2 ref 0 00000001:00080000:0.0:1789760743.179390:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973076 is committed 00000100:00100000:0.0:1789760743.185885:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760743.186048:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495817600 00000100:00100000:2.0:1789760743.186059:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495817600 00000100:00100000:2.0:1789760743.186111:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727430487464 time 727473655863 state 0x0 ref 1 00000100:00100000:2.0:1789760743.186126:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727473916279 time 727473655863 state 0x1 ref 1 00000100:00100000:2.0:1789760743.186159:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727473916279 time 727506874626 state 0x0 ref 1 00000100:00100000:2.0:1789760743.186171:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495817600:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760743.186218:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23592960, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:2.0:1789760743.186228:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454654c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.186245:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760743.186248:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.186254:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454654c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.186266:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454654c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760743.198915:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789760743.198938:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff929036f5aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520307840:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760743.199018:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760743.199265:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520307840 00000100:00100000:3.0:1789760743.199272:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520307840 00000100:00100000:3.0:1789760743.199344:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 200 00000100:00100000:3.0:1789760743.199354:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520307840:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760743.199390:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760743.199614:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520307840:12345-0@lo:13:osp-pre-1-0.0 Request processed in 260us (602us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760743.199622:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 200 00000100:00100000:2.0:1789760743.201574:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff929036f5aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520307840:0@lo:13:osp-pre-1-0.0 00002000:00010000:2.0:1789760743.217391:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23592960 -> 23597056 00010000:00010000:2.0:1789760743.217411:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454654c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.217428:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454654c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.217436:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454654c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.217444:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760743.217449:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454654c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.217457:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454654c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.217476:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454654c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.217483:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454654c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.217576:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495817600:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31402us (31695us total) trans 4294973077 rc 0/0 00000100:00100000:2.0:1789760743.217586:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5789 00000100:00100000:2.0:1789760743.217596:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727473916279 time 727506874626 state 0x2 ref 0 00000001:00080000:1.0:1789760743.217669:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973077 is committed 00000100:00100000:2.0:1789760743.221371:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760743.221519:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495817728 00000100:00100000:1.0:1789760743.221529:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495817728 00000100:00100000:1.0:1789760743.221583:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727473916279 time 727506874626 state 0x0 ref 1 00000100:00100000:1.0:1789760743.221598:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727507135042 time 727506874626 state 0x1 ref 1 00000100:00100000:1.0:1789760743.221630:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727507135042 time 727542346649 state 0x0 ref 1 00000100:00100000:1.0:1789760743.221643:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495817728:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760743.221691:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23597056, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:1.0:1789760743.221700:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546553 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.221715:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760743.221719:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760743.221725:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546553 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.221739:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546553 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760743.247407:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973078 is committed 00002000:00010000:2.0:1789760743.247573:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23597056 -> 23601152 00010000:00010000:2.0:1789760743.247589:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546553 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.247602:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546553 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.247611:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546553 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.247618:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760743.247623:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546553 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.247632:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546553 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.247653:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546553 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.247662:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546553 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.247759:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495817728:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26113us (26401us total) trans 4294973078 rc 0/0 00000100:00100000:2.0:1789760743.247770:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5790 00000100:00100000:2.0:1789760743.247780:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727507135042 time 727542346649 state 0x2 ref 0 00000100:00100000:0.0:1789760743.251042:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760743.251121:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495817856 00000100:00100000:1.0:1789760743.251132:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495817856 00000100:00100000:1.0:1789760743.251190:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727507135042 time 727542346649 state 0x0 ref 1 00000100:00100000:1.0:1789760743.251207:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727542607065 time 727542346649 state 0x1 ref 1 00000100:00100000:1.0:1789760743.251241:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727542607065 time 727571957564 state 0x0 ref 1 00000100:00100000:1.0:1789760743.251253:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495817856:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760743.251336:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23601152, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:1.0:1789760743.251345:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454655a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.251462:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760743.251468:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760743.251475:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454655a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.251490:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454655a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.264037:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760743.280798:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23601152 -> 23605248 00000001:00080000:2.0:1789760743.280803:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973079 is committed 00010000:00010000:1.0:1789760743.280816:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454655a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.280898:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454655a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.280909:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454655a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.280916:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760743.280921:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454655a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.280929:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454655a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.280948:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454655a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.280955:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454655a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760743.281103:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495817856:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29847us (30080us total) trans 4294973079 rc 0/0 00000100:00100000:1.0:1789760743.281113:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5791 00000100:00100000:1.0:1789760743.281123:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727542607065 time 727571957564 state 0x2 ref 0 00000100:00100000:1.0:1789760743.283714:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760743.284014:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495817984 00000100:00100000:2.0:1789760743.284025:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495817984 00000100:00100000:2.0:1789760743.284087:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727542607065 time 727571957564 state 0x0 ref 1 00000100:00100000:2.0:1789760743.284102:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727572217980 time 727571957564 state 0x1 ref 1 00000100:00100000:2.0:1789760743.284135:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727572217980 time 727604851703 state 0x0 ref 1 00000100:00100000:2.0:1789760743.284146:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495817984:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760743.284190:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23605248, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:2.0:1789760743.284198:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546561 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.284213:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760743.284217:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.284223:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546561 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.284234:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546561 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760743.318518:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973080 is committed 00002000:00010000:0.0:1789760743.318628:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23605248 -> 23609344 00010000:00010000:0.0:1789760743.318644:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546561 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.318655:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546561 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.318664:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546561 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.318672:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760743.318677:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546561 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.318687:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546561 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.318707:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546561 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.318714:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546561 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.318807:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495817984:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34657us (35097us total) trans 4294973080 rc 0/0 00000100:00100000:0.0:1789760743.318818:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5792 00000100:00100000:0.0:1789760743.318828:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727572217980 time 727604851703 state 0x2 ref 0 00000100:00100000:2.0:1789760743.322659:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760743.322818:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495818112 00000100:00100000:1.0:1789760743.322827:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495818112 00000100:00100000:1.0:1789760743.322880:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727572217980 time 727604851703 state 0x0 ref 1 00000100:00100000:1.0:1789760743.322893:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727605112119 time 727604851703 state 0x1 ref 1 00000100:00100000:1.0:1789760743.322920:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727605112119 time 727643636776 state 0x0 ref 1 00000100:00100000:1.0:1789760743.322931:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495818112:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760743.322980:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23609344, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:1.0:1789760743.322990:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546568 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.323019:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760743.323024:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760743.323038:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546568 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.323053:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546568 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760743.358919:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23609344 -> 23613440 00010000:00010000:1.0:1789760743.358940:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546568 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.358953:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546568 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.358963:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546568 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.358970:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760743.358975:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546568 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.358984:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546568 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.359018:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546568 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.359036:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546568 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760743.359251:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495818112:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36315us (36594us total) trans 4294973081 rc 0/0 00000100:00100000:1.0:1789760743.359262:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5793 00000100:00100000:1.0:1789760743.359271:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727605112119 time 727643636776 state 0x2 ref 0 00000001:00080000:3.0:1789760743.359475:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973081 is committed 00000100:00100000:1.0:1789760743.367628:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760743.367717:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495818240 00000100:00100000:0.0:1789760743.367727:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495818240 00000100:00100000:0.0:1789760743.367779:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727605112119 time 727643636776 state 0x0 ref 1 00000100:00100000:0.0:1789760743.367793:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727643897192 time 727643636776 state 0x1 ref 1 00000100:00100000:0.0:1789760743.367825:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727643897192 time 727688542517 state 0x0 ref 1 00000100:00100000:0.0:1789760743.367837:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01c8c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495818240:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760743.367882:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23613440, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:0.0:1789760743.367890:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454656f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.367907:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760743.367911:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760743.367919:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454656f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.367933:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454656f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.398517:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973082 is committed 00002000:00010000:1.0:1789760743.398612:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23613440 -> 23617536 00010000:00010000:1.0:1789760743.398626:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454656f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.398638:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454656f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.398648:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454656f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.398655:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760743.398669:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454656f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.398684:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454656f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.398704:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454656f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.398712:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454656f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760743.398806:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01c8c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495818240:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30966us (31183us total) trans 4294973082 rc 0/0 00000100:00100000:1.0:1789760743.398816:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5794 00000100:00100000:1.0:1789760743.398826:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727643897192 time 727688542517 state 0x2 ref 0 00000100:00100000:0.0:1789760743.411976:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760743.412176:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495818368 00000100:00100000:2.0:1789760743.412186:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495818368 00000100:00100000:2.0:1789760743.412242:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727643897192 time 727688542517 state 0x0 ref 1 00000100:00100000:2.0:1789760743.412265:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727688802933 time 727688542517 state 0x1 ref 1 00000100:00100000:2.0:1789760743.412336:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727688802933 time 727733051933 state 0x0 ref 1 00000100:00100000:2.0:1789760743.412349:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495818368:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760743.412396:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23617536, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:2.0:1789760743.412520:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546576 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.412536:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760743.412541:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.412548:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546576 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.412562:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546576 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760743.448466:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973083 is committed 00002000:00010000:2.0:1789760743.448603:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23617536 -> 23621632 00010000:00010000:2.0:1789760743.448618:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546576 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.448630:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546576 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.448638:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546576 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.448652:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760743.448657:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546576 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.448665:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546576 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.448684:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546576 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.448692:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546576 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.448787:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495818368:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36435us (36816us total) trans 4294973083 rc 0/0 00000100:00100000:2.0:1789760743.448797:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5795 00000100:00100000:2.0:1789760743.448807:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727688802933 time 727733051933 state 0x2 ref 0 00000100:00100000:0.0:1789760743.457795:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760743.457893:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495818496 00000100:00100000:1.0:1789760743.457903:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495818496 00000100:00100000:1.0:1789760743.457955:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727688802933 time 727733051933 state 0x0 ref 1 00000100:00100000:1.0:1789760743.457971:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727733312349 time 727733051933 state 0x1 ref 1 00000100:00100000:1.0:1789760743.458121:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727733312349 time 727778722932 state 0x0 ref 1 00000100:00100000:1.0:1789760743.458137:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495818496:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760743.458184:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23621632, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:1.0:1789760743.458194:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454657d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.458210:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760743.458215:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760743.458221:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454657d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.458237:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454657d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.499552:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973084 is committed 00002000:00010000:0.0:1789760743.499788:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23621632 -> 23625728 00010000:00010000:0.0:1789760743.499803:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454657d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.499815:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454657d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.499824:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454657d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.499831:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760743.499836:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454657d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.499845:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454657d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.499865:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454657d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.499873:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454657d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.500362:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495818496:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42221us (42571us total) trans 4294973084 rc 0/0 00000100:00100000:0.0:1789760743.500374:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5796 00000100:00100000:0.0:1789760743.500385:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727733312349 time 727778722932 state 0x2 ref 0 00000100:00100000:1.0:1789760743.504753:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760743.504832:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495818624 00000100:00100000:0.0:1789760743.504841:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495818624 00000100:00100000:0.0:1789760743.504895:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727733312349 time 727778722932 state 0x0 ref 1 00000100:00100000:0.0:1789760743.504911:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727778983348 time 727778722932 state 0x1 ref 1 00000100:00100000:0.0:1789760743.504952:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727778983348 time 727825668507 state 0x0 ref 1 00000100:00100000:0.0:1789760743.504964:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01c8d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495818624:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760743.505007:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23625728, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:0.0:1789760743.505034:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546584 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.505050:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760743.505055:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760743.505061:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546584 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.505075:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546584 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760743.535856:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23625728 -> 23629824 00010000:00010000:0.0:1789760743.535880:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546584 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.535898:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546584 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.535908:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546584 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.535915:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760743.535920:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546584 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.535930:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546584 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.535949:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546584 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.535957:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546584 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.536231:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01c8d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495818624:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31263us (31480us total) trans 4294973085 rc 0/0 00000100:00100000:0.0:1789760743.536242:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5797 00000100:00100000:0.0:1789760743.536252:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727778983348 time 727825668507 state 0x2 ref 0 00000001:00080000:3.0:1789760743.537757:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973085 is committed 00000100:00100000:3.0:1789760743.541560:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760743.541646:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495818752 00000100:00100000:2.0:1789760743.541657:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495818752 00000100:00100000:2.0:1789760743.541721:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727778983348 time 727825668507 state 0x0 ref 1 00000100:00100000:2.0:1789760743.541735:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727825928923 time 727825668507 state 0x1 ref 1 00000100:00100000:2.0:1789760743.541764:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727825928923 time 727862480220 state 0x0 ref 1 00000100:00100000:2.0:1789760743.541780:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495818752:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760743.541829:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23629824, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:2.0:1789760743.541838:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454658b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.541854:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760743.541859:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.541867:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454658b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.541880:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454658b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760743.585954:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973086 is committed 00002000:00010000:2.0:1789760743.586097:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23629824 -> 23633920 00010000:00010000:2.0:1789760743.586113:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454658b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.586125:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454658b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.586134:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454658b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.586140:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760743.586145:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454658b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.586153:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454658b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.586170:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454658b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.586177:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454658b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.586263:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495818752:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44482us (44709us total) trans 4294973086 rc 0/0 00000100:00100000:2.0:1789760743.586274:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5798 00000100:00100000:2.0:1789760743.586309:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727825928923 time 727862480220 state 0x2 ref 0 00000100:00100000:0.0:1789760743.591425:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760743.591537:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495818880 00000100:00100000:1.0:1789760743.591546:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495818880 00000100:00100000:1.0:1789760743.591594:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727825928923 time 727862480220 state 0x0 ref 1 00000100:00100000:1.0:1789760743.591609:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727862740636 time 727862480220 state 0x1 ref 1 00000100:00100000:1.0:1789760743.591640:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727862740636 time 727912356684 state 0x0 ref 1 00000100:00100000:1.0:1789760743.591652:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495818880:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760743.591698:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23633920, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:1.0:1789760743.591708:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546592 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.591724:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760743.591728:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760743.591735:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546592 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.591749:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546592 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760743.619546:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973087 is committed 00002000:00010000:0.0:1789760743.619672:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23633920 -> 23638016 00010000:00010000:0.0:1789760743.619689:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546592 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.619702:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546592 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.619712:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546592 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.619720:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760743.619725:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546592 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.619734:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546592 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.619760:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546592 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.619768:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546592 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.619862:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495818880:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28206us (28441us total) trans 4294973087 rc 0/0 00000100:00100000:0.0:1789760743.619873:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5799 00000100:00100000:0.0:1789760743.619884:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727862740636 time 727912356684 state 0x2 ref 0 00000100:00100000:0.0:1789760743.626352:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760743.626585:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495819008 00000100:00100000:2.0:1789760743.626596:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495819008 00000100:00100000:2.0:1789760743.626649:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727862740636 time 727912356684 state 0x0 ref 1 00000100:00100000:2.0:1789760743.626666:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727912617100 time 727912356684 state 0x1 ref 1 00000100:00100000:2.0:1789760743.626738:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727912617100 time 727947453373 state 0x0 ref 1 00000100:00100000:2.0:1789760743.626750:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495819008:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760743.626801:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23638016, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:2.0:1789760743.626811:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546599 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.626829:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760743.626834:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.626841:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546599 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.626856:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546599 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760743.666361:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973088 is committed 00002000:00010000:0.0:1789760743.666463:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23638016 -> 23642112 00002000:00010000:0.0:1789760743.666468:0:6714:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760742 -> 1789760743 00002000:00010000:0.0:1789760743.666470:0:6714:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760742 -> 1789760743 00010000:00010000:0.0:1789760743.666482:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546599 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.666506:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546599 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.666516:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546599 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.666524:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760743.666529:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546599 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.666538:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546599 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.666557:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546599 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.666565:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546599 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.666654:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495819008:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39902us (40307us total) trans 4294973088 rc 0/0 00000100:00100000:0.0:1789760743.666661:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5800 00000100:00100000:0.0:1789760743.666669:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727912617100 time 727947453373 state 0x2 ref 0 00000100:00100000:1.0:1789760743.671871:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760743.672197:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495819136 00000100:00100000:2.0:1789760743.672207:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495819136 00000100:00100000:2.0:1789760743.672362:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727912617100 time 727947453373 state 0x0 ref 1 00000100:00100000:2.0:1789760743.672379:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727947713789 time 727947453373 state 0x1 ref 1 00000100:00100000:2.0:1789760743.672403:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727947713789 time 727993120440 state 0x0 ref 1 00000100:00100000:2.0:1789760743.672416:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f873480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495819136:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760743.672462:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23642112, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb807 00010000:00010000:2.0:1789760743.672472:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545465a0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.672487:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760743.672491:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.672497:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545465a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.672511:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545465a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760743.716629:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23642112 -> 23646208 00010000:00010000:0.0:1789760743.716648:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545465a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.716660:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545465a0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.716680:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545465a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.716688:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760743.716693:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545465a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.716702:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545465a0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.716725:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545465a0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.716738:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545465a0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.716831:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f873480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495819136:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44413us (44964us total) trans 4294973089 rc 0/0 00000100:00100000:0.0:1789760743.716841:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5801 00000100:00100000:0.0:1789760743.716850:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727947713789 time 727993120440 state 0x2 ref 0 00000001:00080000:2.0:1789760743.717613:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973089 is committed 00000100:00100000:2.0:1789760743.723255:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760743.723361:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495819264 00000100:00100000:0.0:1789760743.723372:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495819264 00000100:00100000:0.0:1789760743.723426:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727947713789 time 727993120440 state 0x0 ref 1 00000100:00100000:0.0:1789760743.723442:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727993380856 time 727993120440 state 0x1 ref 1 00000100:00100000:0.0:1789760743.723474:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727993380856 time 728044191050 state 0x0 ref 1 00000100:00100000:0.0:1789760743.723488:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495819264:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760743.723534:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23646208, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb807 00010000:00010000:0.0:1789760743.723544:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465a7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.723560:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760743.723565:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760743.723572:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.723592:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760743.762397:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973090 is committed 00002000:00010000:0.0:1789760743.762498:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23646208 -> 23650304 00010000:00010000:0.0:1789760743.762512:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.762525:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465a7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.762535:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.762542:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760743.762547:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.762556:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465a7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.762574:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465a7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.762581:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465a7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.762760:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495819264:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39270us (39510us total) trans 4294973090 rc 0/0 00000100:00100000:0.0:1789760743.762771:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5802 00000100:00100000:0.0:1789760743.762781:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727993380856 time 728044191050 state 0x2 ref 0 00000100:00100000:1.0:1789760743.772384:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760743.772830:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495819392 00000100:00100000:0.0:1789760743.772841:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495819392 00000100:00100000:0.0:1789760743.772897:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 727993380856 time 728044191050 state 0x0 ref 1 00000100:00100000:0.0:1789760743.772911:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728044451466 time 728044191050 state 0x1 ref 1 00000100:00100000:0.0:1789760743.772944:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728044451466 time 728093661187 state 0x0 ref 1 00000100:00100000:0.0:1789760743.772956:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495819392:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760743.773006:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23650304, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb807 00010000:00010000:0.0:1789760743.773015:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465ae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.773043:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760743.773048:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760743.773054:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.773070:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760743.816589:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973091 is committed 00002000:00010000:0.0:1789760743.816735:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23650304 -> 23654400 00010000:00010000:0.0:1789760743.816761:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.816774:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465ae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.816783:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.816790:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760743.816795:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.816804:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465ae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.816824:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465ae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.816831:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465ae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.816930:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495819392:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43970us (44550us total) trans 4294973091 rc 0/0 00000100:00100000:0.0:1789760743.816943:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5803 00000100:00100000:0.0:1789760743.816953:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728044451466 time 728093661187 state 0x2 ref 0 00000100:00100000:3.0:1789760743.821545:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760743.821597:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495819520 00000100:00100000:0.0:1789760743.821608:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495819520 00000100:00100000:0.0:1789760743.821666:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728044451466 time 728093661187 state 0x0 ref 1 00000100:00100000:0.0:1789760743.821699:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728093921603 time 728093661187 state 0x1 ref 1 00000100:00100000:0.0:1789760743.821728:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728093921603 time 728142445269 state 0x0 ref 1 00000100:00100000:0.0:1789760743.821752:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495819520:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760743.821801:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23654400, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb807 00010000:00010000:0.0:1789760743.821811:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545465b5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.821827:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760743.821832:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760743.821839:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545465b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.821853:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545465b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760743.861955:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973092 is committed 00002000:00010000:2.0:1789760743.862201:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23654400 -> 23658496 00010000:00010000:2.0:1789760743.862217:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545465b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.862230:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545465b5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.862240:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545465b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.862246:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760743.862250:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545465b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.862259:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545465b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.862301:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545465b5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.862314:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545465b5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.862421:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495819520:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40677us (40880us total) trans 4294973092 rc 0/0 00000100:00100000:2.0:1789760743.862432:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5804 00000100:00100000:2.0:1789760743.862442:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728093921603 time 728142445269 state 0x2 ref 0 00000100:00100000:2.0:1789760743.868116:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760743.868261:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495819648 00000100:00100000:0.0:1789760743.868273:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495819648 00000100:00100000:0.0:1789760743.868372:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728093921603 time 728142445269 state 0x0 ref 1 00000100:00100000:0.0:1789760743.868389:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728142705685 time 728142445269 state 0x1 ref 1 00000100:00100000:0.0:1789760743.868416:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728142705685 time 728189132769 state 0x0 ref 1 00000100:00100000:0.0:1789760743.868429:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495819648:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760743.868487:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23658496, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb807 00010000:00010000:0.0:1789760743.868497:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545465bc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.868514:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760743.868518:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760743.868525:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545465bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.868540:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545465bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760743.896783:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23658496 -> 23662592 00010000:00010000:2.0:1789760743.896806:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545465bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.896819:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545465bc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.896828:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545465bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.896835:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94aa00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760743.896840:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545465bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.896849:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545465bc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.896874:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545465bc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.896881:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545465bc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.896979:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973093 is committed 00000100:00100000:2.0:1789760743.896980:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495819648:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28549us (28868us total) trans 4294973093 rc 0/0 00000100:00100000:2.0:1789760743.896990:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5805 00000100:00100000:2.0:1789760743.897000:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728142705685 time 728189132769 state 0x2 ref 0 00000100:00100000:3.0:1789760743.903432:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760743.905364:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495819776 00000100:00100000:1.0:1789760743.905378:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495819776 00000100:00100000:1.0:1789760743.905439:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728142705685 time 728189132769 state 0x0 ref 1 00000100:00100000:1.0:1789760743.905455:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728189393185 time 728189132769 state 0x1 ref 1 00000100:00100000:1.0:1789760743.905485:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728189393185 time 728226201169 state 0x0 ref 1 00000100:00100000:1.0:1789760743.905499:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495819776:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760743.905545:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23662592, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb807 00010000:00010000:1.0:1789760743.905555:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545465c3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.905572:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760743.905576:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760743.905582:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545465c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.905595:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545465c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.922772:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760743.923176:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495820032 00000100:00100000:0.0:1789760743.923189:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495820032 00000100:00100000:0.0:1789760743.923270:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 485 00000100:00100000:0.0:1789760743.923299:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:17022965-1e4f-46fb-915c-61730096b8e2+8:2359:x1876699495820032:12345-192.168.203.9@tcp:400:kworker.0 00000100:00080000:1.0:1789760743.923438:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760743.925754:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:17022965-1e4f-46fb-915c-61730096b8e2+8:2359:x1876699495820032:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 2459us (2975us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760743.925772:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 485 00000100:00100000:3.0:1789760743.927582:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760743.927645:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760743.927672:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495820160 00000100:00100000:3.0:1789760743.927681:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495820160 00000100:00100000:3.0:1789760743.927740:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 486 00000100:00100000:3.0:1789760743.927750:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2359:x1876699495820160:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:3.0:1789760743.927835:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2359:x1876699495820160:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 86us (189us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760743.927843:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 486 00000100:00080000:3.0:1789760743.927864:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760743.929112:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495819904 00000100:00100000:0.0:1789760743.929124:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495819904 00000100:00100000:0.0:1789760743.929180:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 402 00000100:00100000:0.0:1789760743.929190:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929012311c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2359:x1876699495819904:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:0.0:1789760743.929256:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929012311c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2359:x1876699495819904:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 67us (1684us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760743.929263:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 402 00002000:00010000:0.0:1789760743.947697:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23662592 -> 23666688 00010000:00010000:0.0:1789760743.947715:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545465c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.947728:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545465c3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.947729:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973094 is committed 00010000:00010000:0.0:1789760743.947737:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545465c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.947744:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760743.947749:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545465c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.947758:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545465c3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.947783:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545465c3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.947791:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545465c3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.947892:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495819776:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42391us (44462us total) trans 4294973094 rc 0/0 00000100:00100000:0.0:1789760743.947901:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5806 00000100:00100000:0.0:1789760743.947912:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728189393185 time 728226201169 state 0x2 ref 0 00000100:00100000:3.0:1789760743.952923:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760743.952989:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495820288 00000100:00100000:2.0:1789760743.952999:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495820288 00000100:00100000:2.0:1789760743.953065:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728189393185 time 728226201169 state 0x0 ref 1 00000100:00100000:2.0:1789760743.953079:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728226461585 time 728226201169 state 0x1 ref 1 00000100:00100000:2.0:1789760743.953111:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728226461585 time 728273827568 state 0x0 ref 1 00000100:00100000:2.0:1789760743.953124:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495820288:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760743.953172:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23666688, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb807 00010000:00010000:2.0:1789760743.953181:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545465ca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.953197:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760743.953203:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.953209:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545465ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.953223:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545465ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760743.997211:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23666688 -> 23670784 00010000:00010000:0.0:1789760743.997232:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545465ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.997246:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545465ca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.997254:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545465ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.997262:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760743.997266:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545465ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.997275:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545465ca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.997318:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545465ca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.997332:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545465ca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.997433:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495820288:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44306us (44514us total) trans 4294973095 rc 0/0 00000100:00100000:0.0:1789760743.997444:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5807 00000100:00100000:0.0:1789760743.997454:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728226461585 time 728273827568 state 0x2 ref 0 00000001:00080000:3.0:1789760743.997596:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973095 is committed 00000100:00100000:0.0:1789760744.000682:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760744.000770:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495820416 00000100:00100000:2.0:1789760744.000780:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495820416 00000100:00100000:2.0:1789760744.000835:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728226461585 time 728273827568 state 0x0 ref 1 00000100:00100000:2.0:1789760744.000850:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728274087984 time 728273827568 state 0x1 ref 1 00000100:00100000:2.0:1789760744.000912:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728274087984 time 728321627621 state 0x0 ref 1 00000100:00100000:2.0:1789760744.000926:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495820416:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760744.000971:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23670784, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb807 00010000:00010000:2.0:1789760744.000981:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545465d1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.000996:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760744.001010:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.001017:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545465d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.001040:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545465d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760744.043079:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973096 is committed 00002000:00010000:2.0:1789760744.043155:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23670784 -> 23674880 00010000:00010000:2.0:1789760744.043169:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545465d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.043181:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545465d1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.043190:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545465d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.043196:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760744.043201:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545465d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.043210:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545465d1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.043228:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545465d1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.043235:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545465d1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.043349:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495820416:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42421us (42673us total) trans 4294973096 rc 0/0 00000100:00100000:2.0:1789760744.043362:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5808 00000100:00100000:2.0:1789760744.043372:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728274087984 time 728321627621 state 0x2 ref 0 00000100:00100000:2.0:1789760744.047408:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760744.047616:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495820544 00000100:00100000:0.0:1789760744.047626:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495820544 00000100:00100000:0.0:1789760744.047681:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728274087984 time 728321627621 state 0x0 ref 1 00000100:00100000:0.0:1789760744.047697:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728321888037 time 728321627621 state 0x1 ref 1 00000100:00100000:0.0:1789760744.047731:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728321888037 time 728368442898 state 0x0 ref 1 00000100:00100000:0.0:1789760744.047744:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495820544:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760744.047791:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23674880, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb807 00010000:00010000:0.0:1789760744.047800:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545465d8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.047817:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760744.047822:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.047828:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545465d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.047851:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545465d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760744.097836:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973097 is committed 00002000:00010000:3.0:1789760744.098070:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23674880 -> 23678976 00010000:00010000:3.0:1789760744.098086:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545465d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.098099:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545465d8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.098117:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545465d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.098125:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ae00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760744.098130:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545465d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.098140:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545465d8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.098161:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545465d8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.098173:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545465d8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760744.098264:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495820544:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50517us (50872us total) trans 4294973097 rc 0/0 00000100:00100000:3.0:1789760744.098274:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5809 00000100:00100000:3.0:1789760744.098332:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728321888037 time 728368442898 state 0x2 ref 0 00000100:00100000:2.0:1789760744.103267:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760744.103542:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495820672 00000100:00100000:0.0:1789760744.103553:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495820672 00000100:00100000:0.0:1789760744.103611:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728321888037 time 728368442898 state 0x0 ref 1 00000100:00100000:0.0:1789760744.103627:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728368703314 time 728368442898 state 0x1 ref 1 00000100:00100000:0.0:1789760744.103760:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728368703314 time 728424476233 state 0x0 ref 1 00000100:00100000:0.0:1789760744.103774:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495820672:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760744.103823:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23678976, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb807 00010000:00010000:0.0:1789760744.103834:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545465df lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.103852:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760744.103856:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.103862:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545465df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.103878:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545465df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760744.151980:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23678976 -> 23683072 00010000:00010000:0.0:1789760744.152013:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545465df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.152034:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545465df lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.152044:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545465df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.152051:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760744.152055:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545465df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.152064:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545465df lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.152083:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545465df lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.152091:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545465df lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.152183:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495820672:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48407us (48923us total) trans 4294973098 rc 0/0 00000100:00100000:0.0:1789760744.152192:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5810 00000100:00100000:0.0:1789760744.152202:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728368703314 time 728424476233 state 0x2 ref 0 00000001:00080000:3.0:1789760744.153832:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973098 is committed 00000100:00100000:1.0:1789760744.158003:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760744.158102:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495820800 00000100:00100000:2.0:1789760744.158112:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495820800 00000100:00100000:2.0:1789760744.158167:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728368703314 time 728424476233 state 0x0 ref 1 00000100:00100000:2.0:1789760744.158181:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728424736649 time 728424476233 state 0x1 ref 1 00000100:00100000:2.0:1789760744.158256:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728424736649 time 728478972121 state 0x0 ref 1 00000100:00100000:2.0:1789760744.158270:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495820800:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760744.158334:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23683072, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb807 00010000:00010000:2.0:1789760744.158343:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545465e6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.158369:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760744.158373:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.158380:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545465e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.158395:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545465e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760744.203303:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973099 is committed 00002000:00010000:2.0:1789760744.203418:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23683072 -> 23687168 00010000:00010000:2.0:1789760744.203434:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545465e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.203447:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545465e6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.203456:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545465e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.203613:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760744.203618:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545465e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.203628:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545465e6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.203647:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545465e6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.203656:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545465e6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.203795:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495820800:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45522us (45795us total) trans 4294973099 rc 0/0 00000100:00100000:2.0:1789760744.203808:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5811 00000100:00100000:2.0:1789760744.203818:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728424736649 time 728478972121 state 0x2 ref 0 00000100:00100000:2.0:1789760744.208355:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760744.208452:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495820928 00000100:00100000:0.0:1789760744.208462:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495820928 00000100:00100000:0.0:1789760744.208516:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728424736649 time 728478972121 state 0x0 ref 1 00000100:00100000:0.0:1789760744.208532:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728479232537 time 728478972121 state 0x1 ref 1 00000100:00100000:0.0:1789760744.208563:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728479232537 time 728529279858 state 0x0 ref 1 00000100:00100000:0.0:1789760744.208576:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495820928:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760744.208635:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23687168, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb807 00010000:00010000:0.0:1789760744.208644:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545465ed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.208661:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760744.208666:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.208672:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545465ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.208685:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545465ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760744.264803:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973100 is committed 00002000:00010000:2.0:1789760744.264901:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23687168 -> 23691264 00010000:00010000:2.0:1789760744.264917:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545465ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.264929:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545465ed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.264937:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545465ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.264946:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760744.264951:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545465ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.264960:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545465ed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.264980:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545465ed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.264988:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545465ed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.265104:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495820928:12345-192.168.203.9@tcp:4:dd.0 Request processed in 56525us (56758us total) trans 4294973100 rc 0/0 00000100:00100000:2.0:1789760744.265114:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5812 00000100:00100000:2.0:1789760744.265125:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728479232537 time 728529279858 state 0x2 ref 0 00000100:00100000:3.0:1789760744.269514:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760744.269676:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495821056 00000100:00100000:0.0:1789760744.269687:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495821056 00000100:00100000:0.0:1789760744.269743:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728479232537 time 728529279858 state 0x0 ref 1 00000100:00100000:0.0:1789760744.269758:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728529540274 time 728529279858 state 0x1 ref 1 00000100:00100000:0.0:1789760744.269793:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728529540274 time 728590509458 state 0x0 ref 1 00000100:00100000:0.0:1789760744.269805:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495821056:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760744.269857:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23691264, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb807 00010000:00010000:0.0:1789760744.269866:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465f4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.269882:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760744.269886:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.269893:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.269909:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.292885:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760744.313648:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973101 is committed 00002000:00010000:3.0:1789760744.313710:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23691264 -> 23695360 00010000:00010000:3.0:1789760744.313725:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.313737:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465f4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.313746:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.313753:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760744.313758:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.313773:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465f4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.313792:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465f4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.313800:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545465f4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760744.313890:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495821056:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44082us (44382us total) trans 4294973101 rc 0/0 00000100:00100000:3.0:1789760744.313901:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5813 00000100:00100000:3.0:1789760744.313911:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728529540274 time 728590509458 state 0x2 ref 0 00000100:00100000:3.0:1789760744.322104:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760744.322321:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495821184 00000100:00100000:2.0:1789760744.322330:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495821184 00000100:00100000:2.0:1789760744.322384:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728529540274 time 728590509458 state 0x0 ref 1 00000100:00100000:2.0:1789760744.322398:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728590769874 time 728590509458 state 0x1 ref 1 00000100:00100000:2.0:1789760744.322436:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728590769874 time 728643152483 state 0x0 ref 1 00000100:00100000:2.0:1789760744.322448:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495821184:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760744.322496:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23695360, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb807 00010000:00010000:2.0:1789760744.322505:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545465fb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.322520:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760744.322525:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.322531:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545465fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.322546:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545465fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760744.377166:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973102 is committed 00002000:00010000:0.0:1789760744.377553:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23695360 -> 23699456 00010000:00010000:0.0:1789760744.377569:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545465fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.377581:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545465fb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.377589:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545465fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.377595:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760744.377601:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545465fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.377609:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545465fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.377628:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545465fb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.377635:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545465fb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.377768:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495821184:12345-192.168.203.9@tcp:4:dd.0 Request processed in 55317us (55668us total) trans 4294973102 rc 0/0 00000100:00100000:0.0:1789760744.377780:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5814 00000100:00100000:0.0:1789760744.377789:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728590769874 time 728643152483 state 0x2 ref 0 00000100:00100000:2.0:1789760744.381347:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760744.381641:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495821312 00000100:00100000:0.0:1789760744.381651:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495821312 00000100:00100000:0.0:1789760744.381712:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728590769874 time 728643152483 state 0x0 ref 1 00000100:00100000:0.0:1789760744.381727:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728643412899 time 728643152483 state 0x1 ref 1 00000100:00100000:0.0:1789760744.381758:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728643412899 time 728702474399 state 0x0 ref 1 00000100:00100000:0.0:1789760744.381771:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495821312:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760744.381819:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23699456, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb807 00010000:00010000:0.0:1789760744.381829:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546602 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.381848:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760744.381854:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.381862:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546602 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.381879:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546602 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760744.411882:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23699456 -> 23703552 00010000:00010000:0.0:1789760744.411904:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546602 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.411918:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546602 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.411927:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546602 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.411935:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760744.411939:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546602 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.411948:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546602 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.411973:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546602 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.411981:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546602 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.412092:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495821312:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30318us (30750us total) trans 4294973103 rc 0/0 00000100:00100000:0.0:1789760744.412104:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5815 00000100:00100000:0.0:1789760744.412114:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728643412899 time 728702474399 state 0x2 ref 0 00000001:00080000:2.0:1789760744.412690:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973103 is committed 00000100:00100000:3.0:1789760744.414919:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760744.415110:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495821440 00000100:00100000:0.0:1789760744.415120:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495821440 00000100:00100000:0.0:1789760744.415174:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728643412899 time 728702474399 state 0x0 ref 1 00000100:00100000:0.0:1789760744.415188:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728702734815 time 728702474399 state 0x1 ref 1 00000100:00100000:0.0:1789760744.415224:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728702734815 time 728735940478 state 0x0 ref 1 00000100:00100000:0.0:1789760744.415236:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495821440:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760744.415311:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23703552, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb807 00010000:00010000:0.0:1789760744.415321:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546609 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.415336:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760744.415341:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.415348:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546609 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.415362:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546609 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760744.439746:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973104 is committed 00002000:00010000:3.0:1789760744.439981:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23703552 -> 23707648 00010000:00010000:3.0:1789760744.439995:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546609 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.440008:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546609 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.440017:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546609 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.440494:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760744.440499:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546609 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.440575:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546609 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.440595:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546609 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.440604:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546609 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760744.440769:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495821440:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25530us (25852us total) trans 4294973104 rc 0/0 00000100:00100000:3.0:1789760744.440780:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5816 00000100:00100000:3.0:1789760744.440789:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728702734815 time 728735940478 state 0x2 ref 0 00000100:00100000:1.0:1789760744.445117:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760744.445240:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495821568 00000100:00100000:0.0:1789760744.445249:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495821568 00000100:00100000:0.0:1789760744.445633:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728702734815 time 728735940478 state 0x0 ref 1 00000100:00100000:0.0:1789760744.445652:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728736200894 time 728735940478 state 0x1 ref 1 00000100:00100000:3.0:1789760744.445750:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728736200894 time 728766464112 state 0x0 ref 1 00000100:00100000:3.0:1789760744.445764:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495821568:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760744.445921:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23707648, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb807 00010000:00010000:3.0:1789760744.445931:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546610 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.445952:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760744.445956:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.445963:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546610 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.445976:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546610 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760744.473680:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973105 is committed 00002000:00010000:3.0:1789760744.473782:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23707648 -> 23711744 00010000:00010000:3.0:1789760744.473797:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546610 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.473810:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546610 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.473818:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546610 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.473825:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760744.473830:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546610 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.473838:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546610 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.473857:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546610 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.473866:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546610 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760744.473975:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495821568:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28207us (28860us total) trans 4294973105 rc 0/0 00000100:00100000:3.0:1789760744.473986:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5817 00000100:00100000:3.0:1789760744.475200:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728736200894 time 728766464112 state 0x2 ref 0 00000100:00100000:2.0:1789760744.478121:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760744.478219:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495821696 00000100:00100000:0.0:1789760744.478230:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495821696 00000100:00100000:0.0:1789760744.478317:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728736200894 time 728766464112 state 0x0 ref 1 00000100:00100000:0.0:1789760744.478331:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728766724528 time 728766464112 state 0x1 ref 1 00000100:00100000:0.0:1789760744.478352:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728766724528 time 728799069197 state 0x0 ref 1 00000100:00100000:0.0:1789760744.478365:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495821696:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760744.478411:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23711744, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb807 00010000:00010000:0.0:1789760744.478420:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546617 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.478435:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760744.478440:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.478445:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546617 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.478459:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546617 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760744.509390:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23711744 -> 23715840 00010000:00010000:0.0:1789760744.509411:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546617 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.509504:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546617 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.509514:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546617 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.509521:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760744.509525:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546617 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.509535:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546617 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.509554:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546617 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.509562:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546617 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.509654:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495821696:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31287us (31536us total) trans 4294973106 rc 0/0 00000100:00100000:0.0:1789760744.509665:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5818 00000100:00100000:0.0:1789760744.509674:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728766724528 time 728799069197 state 0x2 ref 0 00000001:00080000:2.0:1789760744.509753:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973106 is committed 00000100:00100000:0.0:1789760744.513759:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760744.513933:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495821824 00000100:00100000:3.0:1789760744.513944:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495821824 00000100:00100000:3.0:1789760744.513996:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728766724528 time 728799069197 state 0x0 ref 1 00000100:00100000:3.0:1789760744.514078:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728799329613 time 728799069197 state 0x1 ref 1 00000100:00100000:3.0:1789760744.514110:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728799329613 time 728834827413 state 0x0 ref 1 00000100:00100000:3.0:1789760744.514122:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495821824:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760744.514170:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23715840, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb807 00010000:00010000:3.0:1789760744.514178:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454661e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.514194:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760744.514199:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.514205:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454661e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.514225:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454661e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760744.552871:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973107 is committed 00002000:00010000:2.0:1789760744.552938:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23715840 -> 23719936 00010000:00010000:2.0:1789760744.552953:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454661e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.552966:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454661e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.552977:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454661e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.552985:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760744.552991:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454661e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.553013:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454661e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.553042:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454661e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.553056:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454661e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.553158:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495821824:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39034us (39403us total) trans 4294973107 rc 0/0 00000100:00100000:2.0:1789760744.553174:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5819 00000100:00100000:2.0:1789760744.553184:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728799329613 time 728834827413 state 0x2 ref 0 00000100:00100000:1.0:1789760744.558765:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760744.558898:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495821952 00000100:00100000:3.0:1789760744.558907:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495821952 00000100:00100000:3.0:1789760744.558955:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728799329613 time 728834827413 state 0x0 ref 1 00000100:00100000:3.0:1789760744.558970:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728835087829 time 728834827413 state 0x1 ref 1 00000100:00100000:3.0:1789760744.558993:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728835087829 time 728879710834 state 0x0 ref 1 00000100:00100000:3.0:1789760744.559015:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495821952:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760744.559066:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23719936, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb807 00010000:00010000:3.0:1789760744.559075:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546625 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.559090:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760744.559094:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.559100:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546625 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.559114:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546625 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760744.617198:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23719936 -> 23724032 00010000:00010000:3.0:1789760744.617437:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546625 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.617451:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546625 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.617473:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546625 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.617481:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760744.617486:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546625 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.617495:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546625 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.617513:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546625 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.617521:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546625 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760744.617606:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495821952:12345-192.168.203.9@tcp:4:dd.0 Request processed in 58588us (58845us total) trans 4294973108 rc 0/0 00000100:00100000:3.0:1789760744.617616:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5820 00000100:00100000:3.0:1789760744.617626:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728835087829 time 728879710834 state 0x2 ref 0 00000001:00080000:2.0:1789760744.618201:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973108 is committed 00000100:00100000:2.0:1789760744.628379:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760744.628849:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495822080 00000100:00100000:0.0:1789760744.628859:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495822080 00000100:00100000:0.0:1789760744.628912:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728835087829 time 728879710834 state 0x0 ref 1 00000100:00100000:0.0:1789760744.628928:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728879971250 time 728879710834 state 0x1 ref 1 00000100:00100000:0.0:1789760744.628957:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728879971250 time 728949673971 state 0x0 ref 1 00000100:00100000:0.0:1789760744.628968:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495822080:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760744.629035:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23724032, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb807 00010000:00010000:0.0:1789760744.629045:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454662c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.629060:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760744.629065:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.629071:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454662c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.629083:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454662c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760744.673845:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973109 is committed 00002000:00010000:2.0:1789760744.674407:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23724032 -> 23728128 00002000:00010000:2.0:1789760744.674412:0:6714:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760743 -> 1789760744 00002000:00010000:2.0:1789760744.674414:0:6714:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760743 -> 1789760744 00010000:00010000:2.0:1789760744.674428:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454662c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.674442:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454662c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.674453:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454662c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.674461:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760744.674466:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454662c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.674477:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454662c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.674496:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454662c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.674503:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454662c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.674597:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495822080:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45627us (46224us total) trans 4294973109 rc 0/0 00000100:00100000:2.0:1789760744.674607:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5821 00000100:00100000:2.0:1789760744.674617:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728879971250 time 728949673971 state 0x2 ref 0 00000100:00100000:1.0:1789760744.680322:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760744.680433:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495822208 00000100:00100000:2.0:1789760744.680443:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495822208 00000100:00100000:2.0:1789760744.680504:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728879971250 time 728949673971 state 0x0 ref 1 00000100:00100000:2.0:1789760744.680519:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728949934387 time 728949673971 state 0x1 ref 1 00000100:00100000:2.0:1789760744.680626:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728949934387 time 729001342240 state 0x0 ref 1 00000100:00100000:2.0:1789760744.680640:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495822208:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760744.680690:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23728128, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb807 00010000:00010000:2.0:1789760744.680708:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546633 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.680724:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760744.680729:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.680735:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546633 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.680751:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546633 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760744.731453:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23728128 -> 23732224 00010000:00010000:3.0:1789760744.731475:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546633 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.731488:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546633 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.731496:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546633 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.731504:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760744.731510:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546633 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.731519:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546633 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.731540:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546633 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.731548:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546633 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760744.731632:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495822208:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50990us (51353us total) trans 4294973110 rc 0/0 00000100:00100000:3.0:1789760744.731643:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5822 00000100:00100000:3.0:1789760744.731654:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728949934387 time 729001342240 state 0x2 ref 0 00000001:00080000:0.0:1789760744.732173:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973110 is committed 00000100:00100000:1.0:1789760744.737047:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760744.737710:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495822336 00000100:00100000:3.0:1789760744.737721:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495822336 00000100:00100000:3.0:1789760744.737777:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 728949934387 time 729001342240 state 0x0 ref 1 00000100:00100000:3.0:1789760744.737793:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729001602656 time 729001342240 state 0x1 ref 1 00000100:00100000:3.0:1789760744.737824:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729001602656 time 729058540826 state 0x0 ref 1 00000100:00100000:3.0:1789760744.737837:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495822336:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760744.737882:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23732224, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb807 00010000:00010000:3.0:1789760744.737891:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454663a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.737908:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760744.737912:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.737919:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454663a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.737933:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454663a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760744.775988:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973111 is committed 00002000:00010000:2.0:1789760744.776077:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23732224 -> 23736320 00010000:00010000:2.0:1789760744.776092:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454663a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.776105:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454663a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.776113:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454663a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.776132:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760744.776137:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454663a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.776145:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454663a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.776165:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454663a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.776172:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454663a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.776265:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495822336:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38426us (39223us total) trans 4294973111 rc 0/0 00000100:00100000:2.0:1789760744.776275:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5823 00000100:00100000:2.0:1789760744.776324:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729001602656 time 729058540826 state 0x2 ref 0 00000100:00100000:2.0:1789760744.784321:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760744.784411:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495822464 00000100:00100000:0.0:1789760744.784494:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495822464 00000100:00100000:0.0:1789760744.784549:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729001602656 time 729058540826 state 0x0 ref 1 00000100:00100000:0.0:1789760744.784565:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729058801242 time 729058540826 state 0x1 ref 1 00000100:00100000:0.0:1789760744.784596:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729058801242 time 729105312902 state 0x0 ref 1 00000100:00100000:0.0:1789760744.784609:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495822464:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760744.784656:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23736320, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb807 00010000:00010000:0.0:1789760744.784665:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546641 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.784681:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760744.784686:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.784692:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546641 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.784705:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546641 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760744.819491:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973112 is committed 00002000:00010000:3.0:1789760744.819606:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23736320 -> 23740416 00010000:00010000:3.0:1789760744.819621:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546641 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.819640:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546641 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.819650:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546641 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.819657:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760744.819663:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546641 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.819672:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546641 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.819694:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546641 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.819702:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546641 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760744.819798:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495822464:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35186us (35513us total) trans 4294973112 rc 0/0 00000100:00100000:3.0:1789760744.819808:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5824 00000100:00100000:3.0:1789760744.819819:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729058801242 time 729105312902 state 0x2 ref 0 00000100:00100000:3.0:1789760744.824577:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760744.825866:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495822592 00000100:00100000:2.0:1789760744.825878:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495822592 00000100:00100000:2.0:1789760744.826158:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729058801242 time 729105312902 state 0x0 ref 1 00000100:00100000:2.0:1789760744.826176:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729105573318 time 729105312902 state 0x1 ref 1 00000100:00100000:2.0:1789760744.826214:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729105573318 time 729146930032 state 0x0 ref 1 00000100:00100000:2.0:1789760744.826228:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495822592:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760744.826301:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23740416, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb807 00010000:00010000:2.0:1789760744.826311:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546648 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.826327:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760744.826331:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.826338:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546648 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.826351:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546648 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760744.858550:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23740416 -> 23744512 00010000:00010000:2.0:1789760744.858569:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546648 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.858587:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546648 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.858596:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546648 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.858604:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760744.858609:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546648 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760744.858613:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973113 is committed 00010000:00010000:2.0:1789760744.858616:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546648 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.858635:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546648 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.858643:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546648 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.858907:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495822592:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32678us (34633us total) trans 4294973113 rc 0/0 00000100:00100000:2.0:1789760744.858922:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5825 00000100:00100000:2.0:1789760744.858933:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729105573318 time 729146930032 state 0x2 ref 0 00000100:00100000:1.0:1789760744.864760:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760744.865068:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495822720 00000100:00100000:2.0:1789760744.865078:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495822720 00000100:00100000:2.0:1789760744.865127:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729105573318 time 729146930032 state 0x0 ref 1 00000100:00100000:2.0:1789760744.865141:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729147190448 time 729146930032 state 0x1 ref 1 00000100:00100000:2.0:1789760744.865173:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729147190448 time 729185890191 state 0x0 ref 1 00000100:00100000:2.0:1789760744.865185:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495822720:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760744.865230:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23744512, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb807 00010000:00010000:2.0:1789760744.865238:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454664f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.865254:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760744.865258:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.865264:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454664f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.865306:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454664f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760744.912131:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973114 is committed 00002000:00010000:0.0:1789760744.912267:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23744512 -> 23748608 00010000:00010000:0.0:1789760744.912327:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454664f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.912570:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454664f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.912581:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454664f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.912590:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760744.912594:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454664f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.912604:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454664f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.912625:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454664f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.912633:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454664f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.912731:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495822720:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47541us (47973us total) trans 4294973114 rc 0/0 00000100:00100000:0.0:1789760744.912743:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5826 00000100:00100000:0.0:1789760744.912752:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729147190448 time 729185890191 state 0x2 ref 0 00000100:00100000:3.0:1789760744.918547:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760744.918626:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495822848 00000100:00100000:0.0:1789760744.918635:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495822848 00000100:00100000:0.0:1789760744.918690:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729147190448 time 729185890191 state 0x0 ref 1 00000100:00100000:0.0:1789760744.918705:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729186150607 time 729185890191 state 0x1 ref 1 00000100:00100000:0.0:1789760744.918738:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729186150607 time 729239455010 state 0x0 ref 1 00000100:00100000:0.0:1789760744.918750:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495822848:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760744.918865:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23748608, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb807 00010000:00010000:0.0:1789760744.918874:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546656 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.918900:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760744.918904:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.918910:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546656 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.918923:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546656 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760744.950833:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973115 is committed 00002000:00010000:3.0:1789760744.950940:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23748608 -> 23752704 00010000:00010000:3.0:1789760744.950957:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546656 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.950971:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546656 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.950980:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546656 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.950988:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760744.951211:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546656 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.951222:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546656 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.951243:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546656 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.951251:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546656 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760744.951506:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495822848:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32752us (32960us total) trans 4294973115 rc 0/0 00000100:00100000:3.0:1789760744.951522:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5827 00000100:00100000:3.0:1789760744.951532:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729186150607 time 729239455010 state 0x2 ref 0 00000100:00100000:3.0:1789760744.954818:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760744.954905:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495822976 00000100:00100000:2.0:1789760744.954922:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495822976 00000100:00100000:2.0:1789760744.954976:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729186150607 time 729239455010 state 0x0 ref 1 00000100:00100000:2.0:1789760744.954992:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729239715426 time 729239455010 state 0x1 ref 1 00000100:00100000:1.0:1789760744.955530:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729239715426 time 729276243662 state 0x0 ref 1 00000100:00100000:0.0:1789760744.955741:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495822976:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760744.955789:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23752704, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb807 00010000:00010000:0.0:1789760744.955798:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454665d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.955814:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760744.955819:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.955825:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454665d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.955839:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454665d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760745.011108:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23752704 -> 23756800 00000001:00080000:1.0:1789760745.011218:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973116 is committed 00010000:00010000:0.0:1789760745.011272:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454665d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.011414:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454665d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.011423:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454665d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.011430:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760745.011435:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454665d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.011444:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454665d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.011462:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454665d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.011474:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454665d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760745.011669:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495822976:12345-192.168.203.9@tcp:4:dd.0 Request processed in 56120us (56854us total) trans 4294973116 rc 0/0 00000100:00100000:0.0:1789760745.011679:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5828 00000100:00100000:0.0:1789760745.011689:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729239715426 time 729276243662 state 0x2 ref 0 00000100:00100000:1.0:1789760745.023525:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760745.025720:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495823104 00000100:00100000:0.0:1789760745.025733:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495823104 00000100:00100000:0.0:1789760745.025876:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729239715426 time 729276243662 state 0x0 ref 1 00000100:00100000:0.0:1789760745.025891:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729276504078 time 729276243662 state 0x1 ref 1 00000100:00100000:0.0:1789760745.025912:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729276504078 time 729346629353 state 0x0 ref 1 00000100:00100000:0.0:1789760745.025925:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04932680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495823104:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760745.025975:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23756800, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb807 00010000:00010000:0.0:1789760745.025983:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546664 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.026010:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760745.026023:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.026030:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546664 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.026049:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546664 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760745.028116:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760745.056851:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973117 is committed 00002000:00010000:2.0:1789760745.056952:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23756800 -> 23760896 00010000:00010000:2.0:1789760745.056970:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546664 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.056985:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546664 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.057006:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546664 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.057023:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760745.057028:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546664 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.057037:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546664 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.057058:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546664 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.057071:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546664 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760745.057368:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04932680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495823104:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31440us (33849us total) trans 4294973117 rc 0/0 00000100:00100000:2.0:1789760745.057378:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5829 00000100:00100000:2.0:1789760745.057389:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729276504078 time 729346629353 state 0x2 ref 0 00000100:00100000:1.0:1789760745.064842:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760745.065074:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495823232 00000100:00100000:3.0:1789760745.065084:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495823232 00000100:00100000:3.0:1789760745.065200:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729276504078 time 729346629353 state 0x0 ref 1 00000100:00100000:3.0:1789760745.065216:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729346889769 time 729346629353 state 0x1 ref 1 00000100:00100000:3.0:1789760745.065322:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729346889769 time 729386038134 state 0x0 ref 1 00000100:00100000:3.0:1789760745.065335:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04930a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495823232:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760745.065387:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23760896, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb807 00010000:00010000:3.0:1789760745.065396:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454666b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.065412:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760745.065416:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.065423:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454666b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.065436:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454666b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760745.100242:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23760896 -> 23764992 00010000:00010000:3.0:1789760745.100261:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454666b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.100275:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454666b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.100299:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454666b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.100307:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760745.100312:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454666b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.100321:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454666b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.100340:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454666b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.100348:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454666b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760745.100420:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973118 is committed 00000100:00100000:3.0:1789760745.100429:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04930a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+6:2359:x1876699495823232:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35092us (35592us total) trans 4294973118 rc 0/0 00000100:00100000:3.0:1789760745.100444:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5830 00000100:00100000:3.0:1789760745.100454:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729346889769 time 729386038134 state 0x2 ref 0 00000100:00100000:1.0:1789760745.111573:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760745.113250:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495823360 00000100:00100000:0.0:1789760745.113263:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495823360 00000100:00100000:0.0:1789760745.113358:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729346889769 time 729386038134 state 0x0 ref 1 00000100:00100000:0.0:1789760745.113374:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729386298550 time 729386038134 state 0x1 ref 1 00000100:00100000:0.0:1789760745.113397:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729386298550 time 729434114199 state 0x0 ref 1 00000100:00100000:0.0:1789760745.113409:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04931180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495823360:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760745.113457:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23764992, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb807 00010000:00010000:0.0:1789760745.113466:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546672 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.113482:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760745.113487:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.113494:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546672 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.113508:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546672 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760745.152837:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23764992 -> 23769088 00010000:00010000:2.0:1789760745.152862:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546672 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.152877:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546672 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.152887:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546672 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.152895:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760745.152900:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546672 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.152910:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546672 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.152936:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546672 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.152946:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546672 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760745.153127:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495823360:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39714us (41558us total) trans 4294973119 rc 0/0 00000100:00100000:2.0:1789760745.153137:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5831 00000100:00100000:2.0:1789760745.153147:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729386298550 time 729434114199 state 0x2 ref 0 00000001:00080000:3.0:1789760745.153726:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973119 is committed 00000100:00100000:2.0:1789760745.164442:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760745.164557:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495823488 00000100:00100000:0.0:1789760745.164566:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495823488 00000100:00100000:0.0:1789760745.164869:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729386298550 time 729434114199 state 0x0 ref 1 00000100:00100000:0.0:1789760745.164886:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729434374615 time 729434114199 state 0x1 ref 1 00000100:00100000:0.0:1789760745.164917:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729434374615 time 729485633624 state 0x0 ref 1 00000100:00100000:0.0:1789760745.164930:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495823488:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760745.164977:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23769088, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb807 00010000:00010000:0.0:1789760745.164986:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546679 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.165002:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760745.165006:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.165025:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546679 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.165039:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546679 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760745.193331:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23769088 -> 23773184 00010000:00010000:0.0:1789760745.193351:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546679 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.193363:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546679 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.193377:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546679 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.193385:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760745.193390:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546679 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.193399:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546679 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.193417:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546679 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.193425:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546679 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760745.193506:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495823488:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28574us (29226us total) trans 4294973120 rc 0/0 00000100:00100000:0.0:1789760745.193516:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5832 00000100:00100000:0.0:1789760745.193525:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729434374615 time 729485633624 state 0x2 ref 0 00000001:00080000:3.0:1789760745.194208:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973120 is committed 00000100:00100000:0.0:1789760745.196697:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760745.196982:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495823616 00000100:00100000:3.0:1789760745.196999:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495823616 00000100:00100000:3.0:1789760745.197059:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729434374615 time 729485633624 state 0x0 ref 1 00000100:00100000:3.0:1789760745.197072:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729485894040 time 729485633624 state 0x1 ref 1 00000100:00100000:3.0:1789760745.197147:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729485894040 time 729517863436 state 0x0 ref 1 00000100:00100000:3.0:1789760745.197160:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495823616:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760745.197204:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23773184, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb807 00010000:00010000:3.0:1789760745.197214:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546680 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.197231:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760745.197235:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.197242:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546680 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.197257:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546680 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760745.243862:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973121 is committed 00002000:00010000:1.0:1789760745.243979:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23773184 -> 23777280 00010000:00010000:1.0:1789760745.244004:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546680 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.244024:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546680 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.244034:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546680 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.244041:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760745.244046:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546680 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.244055:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546680 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.244073:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546680 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.244082:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546680 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760745.244177:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495823616:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47014us (47485us total) trans 4294973121 rc 0/0 00000100:00100000:1.0:1789760745.244188:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5833 00000100:00100000:1.0:1789760745.244198:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729485894040 time 729517863436 state 0x2 ref 0 00000100:00100000:2.0:1789760745.248816:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760745.248890:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495823744 00000100:00100000:0.0:1789760745.248900:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495823744 00000100:00100000:0.0:1789760745.248962:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729485894040 time 729517863436 state 0x0 ref 1 00000100:00100000:0.0:1789760745.248982:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729518123852 time 729517863436 state 0x1 ref 1 00000100:00100000:0.0:1789760745.249033:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729518123852 time 729569749478 state 0x0 ref 1 00000100:00100000:0.0:1789760745.249045:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495823744:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760745.249095:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23777280, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb807 00010000:00010000:0.0:1789760745.249104:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546687 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.249118:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760745.249123:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.249129:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546687 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.249141:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546687 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760745.292767:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973122 is committed 00002000:00010000:2.0:1789760745.292859:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23777280 -> 23781376 00010000:00010000:2.0:1789760745.292875:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546687 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.292890:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546687 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.292900:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546687 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.292907:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760745.292913:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546687 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.292922:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546687 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.292943:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546687 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.292952:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546687 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760745.293093:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495823744:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44045us (44282us total) trans 4294973122 rc 0/0 00000100:00100000:2.0:1789760745.293104:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5834 00000100:00100000:2.0:1789760745.293114:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729518123852 time 729569749478 state 0x2 ref 0 00000100:00100000:2.0:1789760745.300090:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760745.300192:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495823872 00000100:00100000:0.0:1789760745.300203:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495823872 00000100:00100000:0.0:1789760745.300272:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729518123852 time 729569749478 state 0x0 ref 1 00000100:00100000:0.0:1789760745.300326:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729570009894 time 729569749478 state 0x1 ref 1 00000100:00100000:0.0:1789760745.300351:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729570009894 time 729621067867 state 0x0 ref 1 00000100:00100000:0.0:1789760745.300365:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495823872:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760745.300413:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23781376, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb807 00010000:00010000:0.0:1789760745.300423:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454668e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.300438:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760745.300443:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.300450:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454668e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.300464:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454668e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.310825:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760745.344912:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973123 is committed 00002000:00010000:2.0:1789760745.345111:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23781376 -> 23785472 00010000:00010000:2.0:1789760745.345128:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454668e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.345140:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454668e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.345149:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454668e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.345161:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760745.345166:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454668e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.345175:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454668e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.345205:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454668e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.345213:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454668e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760745.345347:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495823872:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44981us (45264us total) trans 4294973123 rc 0/0 00000100:00100000:2.0:1789760745.345357:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5835 00000100:00100000:2.0:1789760745.345367:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729570009894 time 729621067867 state 0x2 ref 0 00000100:00100000:3.0:1789760745.352884:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760745.353123:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495824000 00000100:00100000:1.0:1789760745.353133:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495824000 00000100:00100000:1.0:1789760745.353185:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729570009894 time 729621067867 state 0x0 ref 1 00000100:00100000:1.0:1789760745.353205:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729621328283 time 729621067867 state 0x1 ref 1 00000100:00100000:1.0:1789760745.353362:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729621328283 time 729674077824 state 0x0 ref 1 00000100:00100000:1.0:1789760745.353374:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495824000:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760745.353417:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23785472, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb807 00010000:00010000:1.0:1789760745.353425:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546695 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.353439:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760745.353443:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760745.353449:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546695 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.353462:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546695 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760745.396563:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973124 is committed 00002000:00010000:2.0:1789760745.396776:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23785472 -> 23789568 00010000:00010000:2.0:1789760745.396791:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546695 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.396804:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546695 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.396819:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546695 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.396826:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760745.396831:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546695 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.396840:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546695 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.396862:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546695 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.396871:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546695 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760745.397245:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495824000:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43866us (44363us total) trans 4294973124 rc 0/0 00000100:00100000:2.0:1789760745.397256:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5836 00000100:00100000:2.0:1789760745.397266:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729621328283 time 729674077824 state 0x2 ref 0 00000100:00100000:3.0:1789760745.405931:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760745.406053:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495824128 00000100:00100000:2.0:1789760745.406062:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495824128 00000100:00100000:2.0:1789760745.406113:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729621328283 time 729674077824 state 0x0 ref 1 00000100:00100000:2.0:1789760745.406127:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729674338240 time 729674077824 state 0x1 ref 1 00000100:00100000:2.0:1789760745.406208:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729674338240 time 729726924649 state 0x0 ref 1 00000100:00100000:2.0:1789760745.406222:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495824128:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760745.406269:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23789568, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb807 00010000:00010000:2.0:1789760745.406313:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454669c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.406330:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760745.406334:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760745.406349:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454669c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.406362:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454669c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760745.441479:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23789568 -> 23793664 00010000:00010000:2.0:1789760745.441500:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454669c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.441514:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454669c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.441523:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454669c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.441531:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760745.441537:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454669c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.441545:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454669c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.441565:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454669c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.441573:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454669c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760745.441665:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495824128:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35441us (35739us total) trans 4294973125 rc 0/0 00000100:00100000:2.0:1789760745.441676:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5837 00000100:00100000:2.0:1789760745.441687:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729674338240 time 729726924649 state 0x2 ref 0 00000001:00080000:0.0:1789760745.441845:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973125 is committed 00000100:00100000:3.0:1789760745.452089:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760745.452209:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495824256 00000100:00100000:2.0:1789760745.452220:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495824256 00000100:00100000:2.0:1789760745.452492:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729674338240 time 729726924649 state 0x0 ref 1 00000100:00100000:2.0:1789760745.452513:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729727185065 time 729726924649 state 0x1 ref 1 00000100:00100000:2.0:1789760745.452546:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729727185065 time 729773262931 state 0x0 ref 1 00000100:00100000:2.0:1789760745.452559:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04290700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495824256:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760745.452613:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23793664, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb807 00010000:00010000:2.0:1789760745.452623:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545466a3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.452639:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760745.452643:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760745.452649:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545466a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.452662:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545466a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760745.511762:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973126 is committed 00002000:00010000:3.0:1789760745.511877:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23793664 -> 23797760 00010000:00010000:3.0:1789760745.511894:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545466a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.511908:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545466a3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.511919:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545466a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.511927:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760745.511932:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545466a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.511941:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545466a3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.511960:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545466a3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.511968:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545466a3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.512231:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04290700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495824256:12345-192.168.203.9@tcp:4:dd.0 Request processed in 59667us (60145us total) trans 4294973126 rc 0/0 00000100:00100000:3.0:1789760745.512242:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5838 00000100:00100000:3.0:1789760745.512252:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729727185065 time 729773262931 state 0x2 ref 0 00000100:00100000:1.0:1789760745.524631:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760745.524742:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495824384 00000100:00100000:2.0:1789760745.524751:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495824384 00000100:00100000:2.0:1789760745.524802:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729727185065 time 729773262931 state 0x0 ref 1 00000100:00100000:2.0:1789760745.524817:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729773523347 time 729773262931 state 0x1 ref 1 00000100:00100000:2.0:1789760745.524849:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729773523347 time 729845565090 state 0x0 ref 1 00000100:00100000:2.0:1789760745.524861:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036399050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495824384:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760745.524910:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23797760, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb807 00010000:00010000:2.0:1789760745.524919:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545466aa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.524936:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760745.524940:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760745.524945:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545466aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.524960:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545466aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760745.592039:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23797760 -> 23801856 00010000:00010000:1.0:1789760745.592059:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545466aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.592072:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545466aa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.592087:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545466aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.592094:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760745.592099:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545466aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.592107:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545466aa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.592126:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545466aa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.592135:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545466aa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760745.592221:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036399050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495824384:12345-192.168.203.9@tcp:4:dd.0 Request processed in 67358us (67596us total) trans 4294973127 rc 0/0 00000100:00100000:1.0:1789760745.592232:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5839 00000100:00100000:1.0:1789760745.592242:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729773523347 time 729845565090 state 0x2 ref 0 00000001:00080000:1.0:1789760745.592452:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973127 is committed 00000100:00100000:2.0:1789760745.596162:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760745.596275:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495824512 00000100:00100000:0.0:1789760745.596314:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495824512 00000100:00100000:0.0:1789760745.596367:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729773523347 time 729845565090 state 0x0 ref 1 00000100:00100000:0.0:1789760745.596381:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729845825506 time 729845565090 state 0x1 ref 1 00000100:00100000:0.0:1789760745.596411:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729845825506 time 729917127956 state 0x0 ref 1 00000100:00100000:0.0:1789760745.596434:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495824512:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760745.596480:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23801856, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb807 00010000:00010000:0.0:1789760745.596488:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545466b1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.596510:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760745.596515:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.596521:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545466b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.596534:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545466b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760745.662079:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23801856 -> 23805952 00010000:00010000:1.0:1789760745.662099:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545466b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.662118:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545466b1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.662128:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545466b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.662135:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760745.662140:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545466b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.662150:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545466b1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.662169:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545466b1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.662176:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545466b1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760745.662256:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495824512:12345-192.168.203.9@tcp:4:dd.0 Request processed in 65820us (66098us total) trans 4294973128 rc 0/0 00000100:00100000:1.0:1789760745.662268:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5840 00000100:00100000:1.0:1789760745.662311:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729845825506 time 729917127956 state 0x2 ref 0 00000001:00080000:1.0:1789760745.662353:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973128 is committed 00000100:00100000:2.0:1789760745.666974:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760745.667073:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495824640 00000100:00100000:3.0:1789760745.667083:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495824640 00000100:00100000:3.0:1789760745.667133:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729845825506 time 729917127956 state 0x0 ref 1 00000100:00100000:3.0:1789760745.667146:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729917388372 time 729917127956 state 0x1 ref 1 00000100:00100000:3.0:1789760745.667176:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729917388372 time 729987893373 state 0x0 ref 1 00000100:00100000:3.0:1789760745.667187:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495824640:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760745.667230:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23805952, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb807 00010000:00010000:3.0:1789760745.667238:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545466b8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.667254:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760745.667258:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.667264:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545466b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.667277:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545466b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760745.689571:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973129 is committed 00002000:00010000:3.0:1789760745.689928:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23805952 -> 23810048 00002000:00010000:3.0:1789760745.689933:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760744 -> 1789760745 00002000:00010000:3.0:1789760745.689935:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760744 -> 1789760745 00010000:00010000:3.0:1789760745.689948:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545466b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.689961:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545466b8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.689969:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545466b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.689976:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760745.689982:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545466b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.690099:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545466b8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.690121:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545466b8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.690129:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545466b8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.690434:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495824640:12345-192.168.203.9@tcp:4:dd.0 Request processed in 23244us (23465us total) trans 4294973129 rc 0/0 00000100:00100000:3.0:1789760745.690444:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5841 00000100:00100000:3.0:1789760745.690454:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729917388372 time 729987893373 state 0x2 ref 0 00000100:00100000:3.0:1789760745.702900:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760745.703323:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495824768 00000100:00100000:1.0:1789760745.703336:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495824768 00000100:00100000:1.0:1789760745.703392:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729917388372 time 729987893373 state 0x0 ref 1 00000100:00100000:1.0:1789760745.703406:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729988153789 time 729987893373 state 0x1 ref 1 00000100:00100000:1.0:1789760745.703436:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729988153789 time 730024153186 state 0x0 ref 1 00000100:00100000:1.0:1789760745.703448:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495824768:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760745.703495:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23810048, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb807 00010000:00010000:1.0:1789760745.703504:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545466bf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.703520:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760745.703525:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760745.703532:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545466bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.703547:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545466bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760745.738486:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973130 is committed 00002000:00010000:2.0:1789760745.738813:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23810048 -> 23814144 00010000:00010000:2.0:1789760745.738829:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545466bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.738843:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545466bf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.738853:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545466bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.738861:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760745.738865:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545466bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.738876:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545466bf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.738895:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545466bf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.738903:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545466bf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760745.739103:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495824768:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35651us (36207us total) trans 4294973130 rc 0/0 00000100:00100000:2.0:1789760745.739115:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5842 00000100:00100000:2.0:1789760745.739125:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729988153789 time 730024153186 state 0x2 ref 0 00000100:00100000:0.0:1789760745.748382:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760745.748487:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495824896 00000100:00100000:1.0:1789760745.748496:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495824896 00000100:00100000:1.0:1789760745.748559:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 729988153789 time 730024153186 state 0x0 ref 1 00000100:00100000:1.0:1789760745.748575:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730024413602 time 730024153186 state 0x1 ref 1 00000100:00100000:1.0:1789760745.748611:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730024413602 time 730069327209 state 0x0 ref 1 00000100:00100000:1.0:1789760745.748623:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495824896:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760745.748669:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23814144, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb807 00010000:00010000:1.0:1789760745.748679:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545466c6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.748695:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760745.748699:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760745.748705:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545466c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.748720:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545466c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.761458:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789760745.761481:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff92903fd0ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520307968:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760745.761550:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760745.761699:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520307968 00000100:00100000:0.0:1789760745.761708:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520307968 00000100:00100000:0.0:1789760745.761764:0:6711:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 201 00000100:00100000:0.0:1789760745.761775:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04291180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520307968:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789760745.761799:0:6711:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528416768 pending 1703936 free 3834642432 avail 3810525184 00000100:00100000:0.0:1789760745.761848:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04291180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520307968:12345-0@lo:13:osp-pre-0-0.0 Request processed in 75us (298us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760745.761855:0:6711:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 201 00000100:00080000:3.0:1789760745.761913:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760745.761949:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff92903fd0ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520307968:0@lo:13:osp-pre-0-0.0 00000001:00080000:0.0:1789760745.822419:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973131 is committed 00002000:00010000:1.0:1789760745.822716:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23814144 -> 23818240 00010000:00010000:1.0:1789760745.822733:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545466c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.822746:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545466c6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.822755:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545466c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.822762:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760745.822767:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545466c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.822776:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545466c6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.822796:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545466c6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.822804:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545466c6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760745.822894:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495824896:12345-192.168.203.9@tcp:4:dd.0 Request processed in 74268us (74517us total) trans 4294973131 rc 0/0 00000100:00100000:1.0:1789760745.822905:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5843 00000100:00100000:1.0:1789760745.822916:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730024413602 time 730069327209 state 0x2 ref 0 00000100:00100000:3.0:1789760745.833514:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760745.833685:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495825024 00000100:00100000:0.0:1789760745.833696:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495825024 00000100:00100000:0.0:1789760745.833748:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730024413602 time 730069327209 state 0x0 ref 1 00000100:00100000:0.0:1789760745.833762:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730069587625 time 730069327209 state 0x1 ref 1 00000100:00100000:0.0:1789760745.833789:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730069587625 time 730154506739 state 0x0 ref 1 00000100:00100000:0.0:1789760745.833802:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495825024:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760745.833855:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23818240, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb807 00010000:00010000:0.0:1789760745.833864:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545466cd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.833878:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760745.833881:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.833887:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545466cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.833899:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545466cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760745.872618:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973132 is committed 00002000:00010000:3.0:1789760745.872764:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23818240 -> 23822336 00010000:00010000:3.0:1789760745.872780:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545466cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.872792:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545466cd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.872801:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545466cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.872807:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760745.872813:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545466cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.872826:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545466cd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.872844:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545466cd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.872852:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545466cd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.872940:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495825024:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39135us (39430us total) trans 4294973132 rc 0/0 00000100:00100000:3.0:1789760745.872950:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5844 00000100:00100000:3.0:1789760745.872960:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730069587625 time 730154506739 state 0x2 ref 0 00000100:00100000:3.0:1789760745.883463:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760745.883574:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495825152 00000100:00100000:2.0:1789760745.883595:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495825152 00000100:00100000:2.0:1789760745.883650:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730069587625 time 730154506739 state 0x0 ref 1 00000100:00100000:2.0:1789760745.883672:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730154767155 time 730154506739 state 0x1 ref 1 00000100:00100000:2.0:1789760745.883704:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730154767155 time 730204420340 state 0x0 ref 1 00000100:00100000:2.0:1789760745.883716:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04292a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495825152:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760745.883764:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23822336, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb807 00010000:00010000:2.0:1789760745.883774:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545466d4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.883791:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760745.883795:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760745.883802:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545466d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.883815:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545466d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760745.915890:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973133 is committed 00002000:00010000:1.0:1789760745.916102:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23822336 -> 23826432 00010000:00010000:1.0:1789760745.916118:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545466d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.916130:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545466d4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.916139:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545466d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.916150:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760745.916155:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545466d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.916163:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545466d4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.916249:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545466d4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.916258:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545466d4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760745.916499:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04292a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495825152:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32779us (33040us total) trans 4294973133 rc 0/0 00000100:00100000:1.0:1789760745.916510:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5845 00000100:00100000:1.0:1789760745.916519:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730154767155 time 730204420340 state 0x2 ref 0 00000100:00100000:0.0:1789760745.921641:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760745.921723:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495825280 00000100:00100000:1.0:1789760745.921735:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495825280 00000100:00100000:1.0:1789760745.921786:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730154767155 time 730204420340 state 0x0 ref 1 00000100:00100000:1.0:1789760745.921801:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730204680756 time 730204420340 state 0x1 ref 1 00000100:00100000:1.0:1789760745.921828:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730204680756 time 730242545367 state 0x0 ref 1 00000100:00100000:1.0:1789760745.921843:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495825280:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760745.921887:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23826432, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb807 00010000:00010000:1.0:1789760745.921896:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545466db lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.921912:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760745.921917:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760745.921923:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545466db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.921936:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545466db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760745.955847:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973134 is committed 00002000:00010000:3.0:1789760745.956080:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23826432 -> 23830528 00010000:00010000:3.0:1789760745.956096:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545466db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.956109:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545466db lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.956118:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545466db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.956131:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760745.956136:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545466db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.956145:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545466db lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.956164:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545466db lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.956172:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545466db lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.956267:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495825280:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34421us (34631us total) trans 4294973134 rc 0/0 00000100:00100000:3.0:1789760745.958541:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5846 00000100:00100000:3.0:1789760745.958557:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730204680756 time 730242545367 state 0x2 ref 0 00000100:00100000:3.0:1789760745.960275:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760745.960558:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495825408 00000100:00100000:0.0:1789760745.960567:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495825408 00000100:00100000:0.0:1789760745.960616:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730204680756 time 730242545367 state 0x0 ref 1 00000100:00100000:0.0:1789760745.960631:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730242805783 time 730242545367 state 0x1 ref 1 00000100:00100000:0.0:1789760745.960665:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730242805783 time 730281381725 state 0x0 ref 1 00000100:00100000:0.0:1789760745.960677:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04290700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495825408:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760745.960719:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23830528, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb807 00010000:00010000:0.0:1789760745.960728:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545466e2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.960745:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760745.960749:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.960755:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545466e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.960769:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545466e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760745.990318:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973135 is committed 00002000:00010000:0.0:1789760745.990456:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23830528 -> 23834624 00010000:00010000:0.0:1789760745.990532:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545466e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.990546:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545466e2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.990555:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545466e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.990562:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760745.990567:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545466e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.990576:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545466e2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.990597:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545466e2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.990604:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545466e2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760745.990697:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04290700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495825408:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30017us (30423us total) trans 4294973135 rc 0/0 00000100:00100000:0.0:1789760745.990707:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5847 00000100:00100000:0.0:1789760745.990717:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730242805783 time 730281381725 state 0x2 ref 0 00000100:00100000:1.0:1789760745.994068:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760745.994133:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495825536 00000100:00100000:2.0:1789760745.994142:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495825536 00000100:00100000:2.0:1789760745.994194:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730242805783 time 730281381725 state 0x0 ref 1 00000100:00100000:2.0:1789760745.994209:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730281642141 time 730281381725 state 0x1 ref 1 00000100:00100000:2.0:1789760745.994234:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730281642141 time 730314950868 state 0x0 ref 1 00000100:00100000:2.0:1789760745.994247:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04932680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495825536:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760745.994316:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23834624, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb807 00010000:00010000:2.0:1789760745.994326:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545466e9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.994343:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760745.994348:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760745.994354:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545466e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.994369:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545466e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760746.054852:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23834624 -> 23838720 00010000:00010000:2.0:1789760746.054872:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545466e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.054890:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545466e9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.054899:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545466e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.054906:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760746.054912:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545466e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.054921:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545466e9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.054943:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545466e9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.054950:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545466e9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760746.055060:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04932680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495825536:12345-192.168.203.9@tcp:4:dd.0 Request processed in 60810us (60995us total) trans 4294973136 rc 0/0 00000100:00100000:2.0:1789760746.055077:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5848 00000100:00100000:2.0:1789760746.055087:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730281642141 time 730314950868 state 0x2 ref 0 00000001:00080000:0.0:1789760746.055213:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973136 is committed 00000100:00100000:2.0:1789760746.063245:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760746.063340:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495825664 00000100:00100000:3.0:1789760746.063351:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495825664 00000100:00100000:3.0:1789760746.063403:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730281642141 time 730314950868 state 0x0 ref 1 00000100:00100000:3.0:1789760746.063418:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730315211284 time 730314950868 state 0x1 ref 1 00000100:00100000:3.0:1789760746.063450:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730315211284 time 730384166641 state 0x0 ref 1 00000100:00100000:3.0:1789760746.063462:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495825664:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760746.063508:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23838720, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb807 00010000:00010000:3.0:1789760746.063517:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545466f0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.063533:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760746.063538:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.063544:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545466f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.063557:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545466f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760746.101607:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973137 is committed 00002000:00010000:2.0:1789760746.101728:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23838720 -> 23842816 00010000:00010000:2.0:1789760746.101742:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545466f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.101751:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545466f0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.101758:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545466f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.101762:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760746.101766:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545466f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.101772:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545466f0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.101788:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545466f0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.101793:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545466f0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760746.101978:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495825664:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38512us (38737us total) trans 4294973137 rc 0/0 00000100:00100000:2.0:1789760746.102062:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5849 00000100:00100000:2.0:1789760746.102072:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730315211284 time 730384166641 state 0x2 ref 0 00000100:00100000:3.0:1789760746.108358:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760746.108606:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495825792 00000100:00100000:0.0:1789760746.108616:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495825792 00000100:00100000:0.0:1789760746.108667:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730315211284 time 730384166641 state 0x0 ref 1 00000100:00100000:0.0:1789760746.108693:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730384427057 time 730384166641 state 0x1 ref 1 00000100:00100000:0.0:1789760746.108722:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730384427057 time 730429438737 state 0x0 ref 1 00000100:00100000:0.0:1789760746.108733:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495825792:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760746.108778:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23842816, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb807 00010000:00010000:0.0:1789760746.108786:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545466f7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.108805:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760746.108809:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.108815:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545466f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.108827:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545466f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760746.148230:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23842816 -> 23846912 00010000:00010000:0.0:1789760746.148254:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545466f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.148268:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545466f7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.148277:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545466f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.148311:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760746.148315:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545466f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.148324:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545466f7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.148344:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545466f7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.148353:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545466f7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760746.148450:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495825792:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39713us (40098us total) trans 4294973138 rc 0/0 00000100:00100000:0.0:1789760746.148471:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5850 00000100:00100000:0.0:1789760746.148482:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730384427057 time 730429438737 state 0x2 ref 0 00000001:00080000:2.0:1789760746.149022:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973138 is committed 00000100:00100000:1.0:1789760746.158369:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760746.158490:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495825920 00000100:00100000:2.0:1789760746.158500:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495825920 00000100:00100000:2.0:1789760746.158556:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730384427057 time 730429438737 state 0x0 ref 1 00000100:00100000:2.0:1789760746.158570:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730429699153 time 730429438737 state 0x1 ref 1 00000100:00100000:2.0:1789760746.158607:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730429699153 time 730479323525 state 0x0 ref 1 00000100:00100000:2.0:1789760746.158619:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04930a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495825920:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760746.158682:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23846912, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb807 00010000:00010000:2.0:1789760746.158692:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545466fe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.158708:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760746.158713:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760746.158727:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545466fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.158748:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545466fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760746.211448:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973139 is committed 00002000:00010000:3.0:1789760746.212317:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23846912 -> 23851008 00010000:00010000:3.0:1789760746.212333:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545466fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.212347:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545466fe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.212356:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545466fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.212362:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760746.212367:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545466fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.212377:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545466fe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.212398:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545466fe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.212406:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545466fe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.212505:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04930a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495825920:12345-192.168.203.9@tcp:4:dd.0 Request processed in 53884us (54142us total) trans 4294973139 rc 0/0 00000100:00100000:3.0:1789760746.212515:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5851 00000100:00100000:3.0:1789760746.212525:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730429699153 time 730479323525 state 0x2 ref 0 00000100:00100000:3.0:1789760746.217441:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760746.217847:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495826048 00000100:00100000:2.0:1789760746.217858:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495826048 00000100:00100000:2.0:1789760746.217980:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730429699153 time 730479323525 state 0x0 ref 1 00000100:00100000:2.0:1789760746.217996:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730479583941 time 730479323525 state 0x1 ref 1 00000100:00100000:2.0:1789760746.218033:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730479583941 time 730538750050 state 0x0 ref 1 00000100:00100000:2.0:1789760746.218047:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495826048:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760746.218103:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23851008, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb807 00010000:00010000:2.0:1789760746.218112:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546705 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.218128:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760746.218132:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760746.218138:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546705 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.218154:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546705 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760746.260416:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23851008 -> 23855104 00010000:00010000:3.0:1789760746.260498:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546705 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.260513:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546705 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.260522:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546705 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.260534:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760746.260539:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546705 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.260548:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546705 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.260568:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546705 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.260576:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546705 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.260709:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495826048:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42659us (43272us total) trans 4294973140 rc 0/0 00000100:00100000:3.0:1789760746.260720:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5852 00000100:00100000:3.0:1789760746.260730:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730479583941 time 730538750050 state 0x2 ref 0 00000001:00080000:1.0:1789760746.260926:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973140 is committed 00000100:00100000:2.0:1789760746.278903:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760746.282100:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495826176 00000100:00100000:0.0:1789760746.282115:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495826176 00000100:00100000:0.0:1789760746.282178:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730479583941 time 730538750050 state 0x0 ref 1 00000100:00100000:0.0:1789760746.282194:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730539010466 time 730538750050 state 0x1 ref 1 00000100:00100000:0.0:1789760746.282340:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730539010466 time 730603045626 state 0x0 ref 1 00000100:00100000:0.0:1789760746.282356:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495826176:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760746.282402:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23855104, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb807 00010000:00010000:0.0:1789760746.282412:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454670c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.282427:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760746.282431:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.282438:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454670c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.282453:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454670c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760746.322803:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23855104 -> 23859200 00010000:00010000:0.0:1789760746.322822:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454670c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.322835:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454670c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.322843:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454670c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.322850:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760746.322854:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454670c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.322863:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454670c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.322882:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454670c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.322889:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454670c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760746.322995:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495826176:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40631us (44088us total) trans 4294973141 rc 0/0 00000100:00100000:0.0:1789760746.323018:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5853 00000100:00100000:0.0:1789760746.323029:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730539010466 time 730603045626 state 0x2 ref 0 00000001:00080000:1.0:1789760746.323918:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973141 is committed 00000100:00100000:0.0:1789760746.328415:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760746.331236:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495826304 00000100:00100000:3.0:1789760746.331250:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495826304 00000100:00100000:3.0:1789760746.331337:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730539010466 time 730603045626 state 0x0 ref 1 00000100:00100000:3.0:1789760746.331353:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730603306042 time 730603045626 state 0x1 ref 1 00000100:00100000:3.0:1789760746.331389:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730603306042 time 730652105636 state 0x0 ref 1 00000100:00100000:3.0:1789760746.331407:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92901237ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495826304:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760746.331461:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23859200, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb807 00010000:00010000:3.0:1789760746.331469:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546713 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.331626:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760746.331631:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.331638:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546713 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.331653:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546713 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.334845:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760746.355423:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973142 is committed 00002000:00010000:3.0:1789760746.355630:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23859200 -> 23863296 00010000:00010000:3.0:1789760746.355645:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546713 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.355659:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546713 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.355667:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546713 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.355675:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760746.355681:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546713 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.355691:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546713 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.355709:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546713 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.355716:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546713 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.355868:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92901237ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495826304:12345-192.168.203.9@tcp:4:dd.0 Request processed in 24459us (27458us total) trans 4294973142 rc 0/0 00000100:00100000:3.0:1789760746.355878:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5854 00000100:00100000:3.0:1789760746.355888:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730603306042 time 730652105636 state 0x2 ref 0 00000100:00100000:2.0:1789760746.358903:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760746.358976:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495826432 00000100:00100000:3.0:1789760746.358994:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495826432 00000100:00100000:3.0:1789760746.359054:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730603306042 time 730652105636 state 0x0 ref 1 00000100:00100000:3.0:1789760746.359070:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730652366052 time 730652105636 state 0x1 ref 1 00000100:00100000:3.0:1789760746.359090:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730652366052 time 730679807729 state 0x0 ref 1 00000100:00100000:3.0:1789760746.359102:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495826432:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760746.359146:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23863296, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb807 00010000:00010000:3.0:1789760746.359155:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454671a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.359171:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760746.359175:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.359181:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454671a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.359194:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454671a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.387228:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973143 is committed 00002000:00010000:3.0:1789760746.387435:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23863296 -> 23867392 00010000:00010000:3.0:1789760746.387455:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454671a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.387469:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454671a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.387479:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454671a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.387487:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760746.387492:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454671a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.387501:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454671a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.387520:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454671a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.387529:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454671a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.387661:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495826432:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28556us (28761us total) trans 4294973143 rc 0/0 00000100:00100000:3.0:1789760746.387672:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5855 00000100:00100000:3.0:1789760746.387683:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730652366052 time 730679807729 state 0x2 ref 0 00000100:00100000:2.0:1789760746.391953:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760746.392233:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495826560 00000100:00100000:0.0:1789760746.392243:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495826560 00000100:00100000:0.0:1789760746.392512:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730652366052 time 730679807729 state 0x0 ref 1 00000100:00100000:0.0:1789760746.392527:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730680068145 time 730679807729 state 0x1 ref 1 00000100:00100000:0.0:1789760746.392882:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730680068145 time 730713274967 state 0x0 ref 1 00000100:00100000:0.0:1789760746.393064:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495826560:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760746.393192:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23867392, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb807 00010000:00010000:0.0:1789760746.393202:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546721 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.393220:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760746.393226:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.393233:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546721 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.393246:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546721 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760746.424539:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973144 is committed 00002000:00010000:2.0:1789760746.424637:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23867392 -> 23871488 00010000:00010000:2.0:1789760746.424652:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546721 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.424665:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546721 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.424680:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546721 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.424687:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760746.424692:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546721 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.424701:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546721 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.424719:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546721 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.424727:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546721 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760746.424816:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495826560:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31917us (32869us total) trans 4294973144 rc 0/0 00000100:00100000:2.0:1789760746.424826:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5856 00000100:00100000:2.0:1789760746.424836:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730680068145 time 730713274967 state 0x2 ref 0 00000100:00100000:1.0:1789760746.427590:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760746.428056:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495826688 00000100:00100000:3.0:1789760746.428066:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495826688 00000100:00100000:3.0:1789760746.428121:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730680068145 time 730713274967 state 0x0 ref 1 00000100:00100000:3.0:1789760746.428135:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730713535383 time 730713274967 state 0x1 ref 1 00000100:00100000:3.0:1789760746.428205:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730713535383 time 730748921642 state 0x0 ref 1 00000100:00100000:3.0:1789760746.428219:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04931180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495826688:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760746.428266:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23871488, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb807 00010000:00010000:3.0:1789760746.428275:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546728 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.428462:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760746.428467:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.428473:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546728 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.428487:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546728 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.459215:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973145 is committed 00002000:00010000:0.0:1789760746.459321:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23871488 -> 23875584 00010000:00010000:0.0:1789760746.459411:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546728 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.459425:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546728 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.459434:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546728 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.459442:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760746.459447:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546728 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.459457:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546728 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.459477:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546728 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.459486:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546728 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760746.459619:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495826688:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31397us (32032us total) trans 4294973145 rc 0/0 00000100:00100000:0.0:1789760746.459634:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5857 00000100:00100000:0.0:1789760746.459645:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730713535383 time 730748921642 state 0x2 ref 0 00000100:00100000:3.0:1789760746.463735:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760746.463832:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495826816 00000100:00100000:2.0:1789760746.463842:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495826816 00000100:00100000:2.0:1789760746.463894:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730713535383 time 730748921642 state 0x0 ref 1 00000100:00100000:2.0:1789760746.463912:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730749182058 time 730748921642 state 0x1 ref 1 00000100:00100000:2.0:1789760746.463944:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730749182058 time 730784660656 state 0x0 ref 1 00000100:00100000:2.0:1789760746.463956:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495826816:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760746.464002:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23875584, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb807 00010000:00010000:2.0:1789760746.464021:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454672f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.464036:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760746.464040:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760746.464046:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454672f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.464059:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454672f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760746.494580:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973146 is committed 00002000:00010000:2.0:1789760746.494743:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23875584 -> 23879680 00010000:00010000:2.0:1789760746.494759:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454672f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.494773:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454672f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.494783:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454672f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.494794:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760746.494799:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454672f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.494808:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454672f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.494828:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454672f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.494836:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454672f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760746.494928:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495826816:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30968us (31197us total) trans 4294973146 rc 0/0 00000100:00100000:2.0:1789760746.494938:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5858 00000100:00100000:2.0:1789760746.494948:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730749182058 time 730784660656 state 0x2 ref 0 00000100:00100000:2.0:1789760746.505020:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760746.505170:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495826944 00000100:00100000:0.0:1789760746.505180:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495826944 00000100:00100000:0.0:1789760746.505236:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730749182058 time 730784660656 state 0x0 ref 1 00000100:00100000:0.0:1789760746.505251:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730784921072 time 730784660656 state 0x1 ref 1 00000100:00100000:0.0:1789760746.505369:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730784921072 time 730826084639 state 0x0 ref 1 00000100:00100000:0.0:1789760746.505382:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495826944:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760746.505431:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23879680, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb807 00010000:00010000:0.0:1789760746.505440:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546736 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.505456:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760746.505461:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.505468:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546736 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.505492:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546736 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760746.547722:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973147 is committed 00002000:00010000:0.0:1789760746.547836:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23879680 -> 23883776 00010000:00010000:0.0:1789760746.547853:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546736 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.547866:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546736 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.547886:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546736 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.547895:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760746.547900:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546736 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.547909:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546736 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.547930:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546736 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.547939:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546736 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760746.548170:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495826944:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42784us (43162us total) trans 4294973147 rc 0/0 00000100:00100000:0.0:1789760746.548181:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5859 00000100:00100000:0.0:1789760746.548190:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730784921072 time 730826084639 state 0x2 ref 0 00000100:00100000:3.0:1789760746.551497:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760746.551679:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495827072 00000100:00100000:0.0:1789760746.551689:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495827072 00000100:00100000:0.0:1789760746.551742:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730784921072 time 730826084639 state 0x0 ref 1 00000100:00100000:0.0:1789760746.551756:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730826345055 time 730826084639 state 0x1 ref 1 00000100:00100000:0.0:1789760746.551788:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730826345055 time 730872504738 state 0x0 ref 1 00000100:00100000:0.0:1789760746.551801:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495827072:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760746.551852:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23883776, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb807 00010000:00010000:0.0:1789760746.551863:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454673d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.551892:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760746.551898:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.551906:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454673d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.551920:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454673d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760746.582739:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973148 is committed 00002000:00010000:0.0:1789760746.582839:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23883776 -> 23887872 00010000:00010000:0.0:1789760746.582855:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454673d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.582868:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454673d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.582957:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454673d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.582965:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760746.582970:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454673d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.582988:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454673d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.583017:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454673d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.583025:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454673d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760746.583114:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495827072:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31312us (31623us total) trans 4294973148 rc 0/0 00000100:00100000:0.0:1789760746.583124:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5860 00000100:00100000:0.0:1789760746.583134:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730826345055 time 730872504738 state 0x2 ref 0 00000100:00100000:3.0:1789760746.586651:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760746.586828:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495827200 00000100:00100000:2.0:1789760746.586838:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495827200 00000100:00100000:2.0:1789760746.586898:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730826345055 time 730872504738 state 0x0 ref 1 00000100:00100000:2.0:1789760746.586913:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730872765154 time 730872504738 state 0x1 ref 1 00000100:00100000:2.0:1789760746.586994:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730872765154 time 730907710308 state 0x0 ref 1 00000100:00100000:2.0:1789760746.587020:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495827200:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760746.587080:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23887872, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb807 00010000:00010000:2.0:1789760746.587089:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546744 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.587204:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760746.587209:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760746.587216:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546744 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.587230:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546744 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760746.617926:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973149 is committed 00002000:00010000:2.0:1789760746.618129:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23887872 -> 23891968 00010000:00010000:2.0:1789760746.618145:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546744 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.618158:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546744 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.618167:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546744 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.618175:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760746.618180:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546744 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.618190:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546744 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.618210:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546744 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.618218:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546744 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760746.618429:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495827200:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31418us (31782us total) trans 4294973149 rc 0/0 00000100:00100000:2.0:1789760746.618440:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5861 00000100:00100000:2.0:1789760746.618450:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730872765154 time 730907710308 state 0x2 ref 0 00000100:00100000:3.0:1789760746.629997:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760746.630066:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495827328 00000100:00100000:0.0:1789760746.630076:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495827328 00000100:00100000:0.0:1789760746.630129:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730872765154 time 730907710308 state 0x0 ref 1 00000100:00100000:0.0:1789760746.630144:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730907970724 time 730907710308 state 0x1 ref 1 00000100:00100000:0.0:1789760746.630173:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730907970724 time 730950890122 state 0x0 ref 1 00000100:00100000:0.0:1789760746.630187:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495827328:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760746.630236:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23891968, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb807 00010000:00010000:0.0:1789760746.630245:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454674b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.630262:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760746.630266:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.630274:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454674b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.630325:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454674b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760746.680473:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973150 is committed 00002000:00010000:3.0:1789760746.680965:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23891968 -> 23896064 00002000:00010000:3.0:1789760746.680969:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760745 -> 1789760746 00002000:00010000:3.0:1789760746.680972:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760745 -> 1789760746 00010000:00010000:3.0:1789760746.680997:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454674b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.681018:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454674b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.681028:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454674b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.681036:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760746.681040:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454674b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.681054:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454674b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.681075:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454674b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.681082:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454674b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.681207:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495827328:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51019us (51215us total) trans 4294973150 rc 0/0 00000100:00100000:3.0:1789760746.681218:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5862 00000100:00100000:3.0:1789760746.681228:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730907970724 time 730950890122 state 0x2 ref 0 00000100:00100000:1.0:1789760746.686765:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760746.686861:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495827456 00000100:00100000:0.0:1789760746.686871:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495827456 00000100:00100000:0.0:1789760746.686925:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730907970724 time 730950890122 state 0x0 ref 1 00000100:00100000:0.0:1789760746.686940:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730951150538 time 730950890122 state 0x1 ref 1 00000100:00100000:0.0:1789760746.687069:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730951150538 time 731007784573 state 0x0 ref 1 00000100:00100000:0.0:1789760746.687083:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04931f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495827456:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760746.687129:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23896064, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb807 00010000:00010000:0.0:1789760746.687139:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546752 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.687155:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760746.687160:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.687167:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546752 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.687181:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546752 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760746.715969:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973151 is committed 00002000:00010000:0.0:1789760746.716097:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23896064 -> 23900160 00010000:00010000:0.0:1789760746.716112:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546752 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.716125:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546752 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.716135:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546752 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.716142:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760746.716147:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546752 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.716157:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546752 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.716177:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546752 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.716185:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546752 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760746.716274:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495827456:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29189us (29513us total) trans 4294973151 rc 0/0 00000100:00100000:0.0:1789760746.719061:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5863 00000100:00100000:0.0:1789760746.719081:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730951150538 time 731007784573 state 0x2 ref 0 00000100:00100000:3.0:1789760746.719691:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760746.719928:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495827584 00000100:00100000:1.0:1789760746.719937:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495827584 00000100:00100000:1.0:1789760746.719992:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 730951150538 time 731007784573 state 0x0 ref 1 00000100:00100000:1.0:1789760746.720014:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731008044989 time 731007784573 state 0x1 ref 1 00000100:00100000:1.0:1789760746.720036:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731008044989 time 731040753733 state 0x0 ref 1 00000100:00100000:1.0:1789760746.720049:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495827584:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760746.720093:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23900160, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb807 00010000:00010000:1.0:1789760746.720101:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546759 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.720116:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760746.720121:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760746.720127:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546759 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.720141:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546759 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.761654:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973152 is committed 00002000:00010000:1.0:1789760746.761758:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23900160 -> 23904256 00010000:00010000:1.0:1789760746.761834:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546759 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.761847:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546759 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.761856:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546759 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.761863:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760746.761868:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546759 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.762066:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546759 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.762086:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546759 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.762095:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546759 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.762226:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495827584:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42174us (42532us total) trans 4294973152 rc 0/0 00000100:00100000:1.0:1789760746.762238:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5864 00000100:00100000:1.0:1789760746.762248:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731008044989 time 731040753733 state 0x2 ref 0 00000100:00100000:3.0:1789760746.767250:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760746.768085:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495827712 00000100:00100000:0.0:1789760746.768097:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495827712 00000100:00100000:0.0:1789760746.768154:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731008044989 time 731040753733 state 0x0 ref 1 00000100:00100000:0.0:1789760746.768169:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731041014149 time 731040753733 state 0x1 ref 1 00000100:00100000:0.0:1789760746.768238:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731041014149 time 731088954258 state 0x0 ref 1 00000100:00100000:0.0:1789760746.768252:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495827712:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760746.768332:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23904256, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb807 00010000:00010000:0.0:1789760746.768341:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546760 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.768357:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760746.768362:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.768369:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546760 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.768383:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546760 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760746.821880:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973153 is committed 00002000:00010000:2.0:1789760746.822596:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23904256 -> 23908352 00010000:00010000:2.0:1789760746.822613:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546760 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.822626:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546760 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.822636:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546760 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.822649:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760746.822654:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546760 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.822663:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546760 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.822682:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546760 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.822690:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546760 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760746.822917:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495827712:12345-192.168.203.9@tcp:4:dd.0 Request processed in 54663us (55671us total) trans 4294973153 rc 0/0 00000100:00100000:2.0:1789760746.822928:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5865 00000100:00100000:2.0:1789760746.822938:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731041014149 time 731088954258 state 0x2 ref 0 00000100:00100000:3.0:1789760746.827738:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760746.827817:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495827840 00000100:00100000:0.0:1789760746.827826:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495827840 00000100:00100000:0.0:1789760746.827878:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731041014149 time 731088954258 state 0x0 ref 1 00000100:00100000:0.0:1789760746.827893:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731089214674 time 731088954258 state 0x1 ref 1 00000100:00100000:0.0:1789760746.827919:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731089214674 time 731148636031 state 0x0 ref 1 00000100:00100000:0.0:1789760746.827931:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495827840:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760746.827980:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23908352, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb807 00010000:00010000:0.0:1789760746.828008:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546767 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.828077:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760746.828082:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.828088:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546767 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.828103:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546767 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760746.876319:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973154 is committed 00002000:00010000:2.0:1789760746.876473:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23908352 -> 23912448 00010000:00010000:2.0:1789760746.876488:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546767 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.876503:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546767 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.876518:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546767 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.876527:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760746.876532:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546767 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.876542:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546767 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.876560:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546767 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.876567:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546767 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760746.876653:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495827840:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48719us (48919us total) trans 4294973154 rc 0/0 00000100:00100000:2.0:1789760746.876662:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5866 00000100:00100000:2.0:1789760746.876672:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731089214674 time 731148636031 state 0x2 ref 0 00000100:00100000:0.0:1789760746.882518:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760746.882620:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495827968 00000100:00100000:1.0:1789760746.882629:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495827968 00000100:00100000:1.0:1789760746.882684:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731089214674 time 731148636031 state 0x0 ref 1 00000100:00100000:1.0:1789760746.882698:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731148896447 time 731148636031 state 0x1 ref 1 00000100:00100000:1.0:1789760746.882728:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731148896447 time 731203443784 state 0x0 ref 1 00000100:00100000:1.0:1789760746.882741:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03095500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495827968:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760746.882793:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23912448, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb807 00010000:00010000:1.0:1789760746.882802:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454676e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.882820:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760746.882824:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760746.882832:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454676e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.882846:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454676e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760746.921374:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973155 is committed 00002000:00010000:2.0:1789760746.921686:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23912448 -> 23916544 00010000:00010000:2.0:1789760746.921700:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454676e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.921718:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454676e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.921726:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454676e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.921733:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760746.921737:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454676e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.921745:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454676e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.921764:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454676e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.921771:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454676e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760746.921856:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495827968:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39114us (39347us total) trans 4294973155 rc 0/0 00000100:00100000:2.0:1789760746.921866:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5867 00000100:00100000:2.0:1789760746.921876:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731148896447 time 731203443784 state 0x2 ref 0 00000100:00100000:3.0:1789760746.925859:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760746.926695:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495828096 00000100:00100000:0.0:1789760746.926705:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495828096 00000100:00100000:0.0:1789760746.926758:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731148896447 time 731203443784 state 0x0 ref 1 00000100:00100000:0.0:1789760746.926772:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731203704200 time 731203443784 state 0x1 ref 1 00000100:00100000:0.0:1789760746.926835:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731203704200 time 731247552529 state 0x0 ref 1 00000100:00100000:0.0:1789760746.926848:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495828096:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760746.926894:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23916544, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb807 00010000:00010000:0.0:1789760746.926902:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546775 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.926917:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760746.926922:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.926928:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546775 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.926940:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546775 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760746.959178:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973156 is committed 00002000:00010000:3.0:1789760746.959353:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23916544 -> 23920640 00010000:00010000:3.0:1789760746.959368:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546775 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.959380:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546775 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.959389:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546775 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.959397:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760746.959402:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546775 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.959411:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546775 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.959431:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546775 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.959438:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546775 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.959522:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495828096:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32671us (33666us total) trans 4294973156 rc 0/0 00000100:00100000:3.0:1789760746.959532:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5868 00000100:00100000:3.0:1789760746.959541:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731203704200 time 731247552529 state 0x2 ref 0 00000100:00100000:3.0:1789760746.964839:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760746.968176:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495828224 00000100:00100000:2.0:1789760746.968192:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495828224 00000100:00100000:2.0:1789760746.968260:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731203704200 time 731247552529 state 0x0 ref 1 00000100:00100000:2.0:1789760746.968306:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731247812945 time 731247552529 state 0x1 ref 1 00000100:00100000:2.0:1789760746.968347:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731247812945 time 731289063634 state 0x0 ref 1 00000100:00100000:2.0:1789760746.968360:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495828224:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760746.968409:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23920640, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb807 00010000:00010000:2.0:1789760746.968419:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454677c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.968436:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760746.968440:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760746.968447:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454677c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.968462:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454677c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760747.000229:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973157 is committed 00002000:00010000:2.0:1789760747.000743:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23920640 -> 23924736 00010000:00010000:2.0:1789760747.000759:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454677c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.000772:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454677c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.000787:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454677c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.000796:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760747.000801:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454677c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.000811:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454677c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.000833:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454677c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.000840:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454677c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.000941:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495828224:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32577us (36106us total) trans 4294973157 rc 0/0 00000100:00100000:2.0:1789760747.000952:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5869 00000100:00100000:2.0:1789760747.000962:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731247812945 time 731289063634 state 0x2 ref 0 00000100:00100000:1.0:1789760747.004862:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760747.004936:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495828352 00000100:00100000:2.0:1789760747.004945:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495828352 00000100:00100000:2.0:1789760747.005134:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731247812945 time 731289063634 state 0x0 ref 1 00000100:00100000:2.0:1789760747.005148:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731289324050 time 731289063634 state 0x1 ref 1 00000100:00100000:2.0:1789760747.005178:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731289324050 time 731325895672 state 0x0 ref 1 00000100:00100000:2.0:1789760747.005190:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495828352:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760747.005236:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23924736, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb807 00010000:00010000:2.0:1789760747.005245:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546783 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.005260:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760747.005264:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.005271:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546783 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.005308:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546783 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760747.005974:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789760747.052867:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23924736 -> 23928832 00010000:00010000:0.0:1789760747.052890:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546783 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.052910:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546783 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.052920:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546783 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.052928:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760747.052934:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546783 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.052943:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546783 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.052966:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546783 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.052974:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546783 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.053148:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495828352:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47954us (48288us total) trans 4294973158 rc 0/0 00000100:00100000:0.0:1789760747.053159:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5870 00000100:00100000:0.0:1789760747.053171:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731289324050 time 731325895672 state 0x2 ref 0 00000100:00100000:0.0:1789760747.057430:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000001:00080000:2.0:1789760747.057689:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973158 is committed 00000100:00100000:3.0:1789760747.057964:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495828480 00000100:00100000:3.0:1789760747.057973:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495828480 00000100:00100000:3.0:1789760747.058149:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731289324050 time 731325895672 state 0x0 ref 1 00000100:00100000:3.0:1789760747.058165:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731326156088 time 731325895672 state 0x1 ref 1 00000100:00100000:3.0:1789760747.058241:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731326156088 time 731378956292 state 0x0 ref 1 00000100:00100000:3.0:1789760747.058254:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03095180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495828480:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760747.058340:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23928832, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb807 00010000:00010000:3.0:1789760747.058350:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454678a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.058368:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760747.058374:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.058381:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454678a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.058397:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454678a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760747.094118:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23928832 -> 23932928 00010000:00010000:0.0:1789760747.094138:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454678a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.094153:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454678a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.094162:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454678a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.094169:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760747.094174:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454678a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.094183:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454678a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.094203:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454678a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.094211:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454678a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760747.094621:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973159 is committed 00000100:00100000:0.0:1789760747.095555:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495828480:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37295us (38129us total) trans 4294973159 rc 0/0 00000100:00100000:0.0:1789760747.095567:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5871 00000100:00100000:0.0:1789760747.095576:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731326156088 time 731378956292 state 0x2 ref 0 00000100:00100000:1.0:1789760747.103459:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760747.104571:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495828608 00000100:00100000:2.0:1789760747.104582:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495828608 00000100:00100000:2.0:1789760747.104648:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731326156088 time 731378956292 state 0x0 ref 1 00000100:00100000:2.0:1789760747.104663:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731379216708 time 731378956292 state 0x1 ref 1 00000100:00100000:2.0:1789760747.104737:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731379216708 time 731425453547 state 0x0 ref 1 00000100:00100000:2.0:1789760747.104751:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495828608:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760747.104800:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23932928, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb807 00010000:00010000:2.0:1789760747.104809:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546791 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.104824:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760747.104829:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.104834:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546791 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.104847:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546791 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760747.155754:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973160 is committed 00002000:00010000:0.0:1789760747.155855:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23932928 -> 23937024 00010000:00010000:0.0:1789760747.155872:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546791 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.155884:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546791 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.155892:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546791 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.155899:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760747.155903:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546791 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.155911:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546791 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.155932:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546791 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.155940:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546791 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.156191:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495828608:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51438us (52737us total) trans 4294973160 rc 0/0 00000100:00100000:0.0:1789760747.156202:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5872 00000100:00100000:0.0:1789760747.156213:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731379216708 time 731425453547 state 0x2 ref 0 00000100:00100000:1.0:1789760747.160111:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760747.161424:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495828736 00000100:00100000:0.0:1789760747.161437:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495828736 00000100:00100000:1.0:1789760747.161826:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731379216708 time 731425453547 state 0x0 ref 1 00000100:00100000:1.0:1789760747.161841:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731425713963 time 731425453547 state 0x1 ref 1 00000100:00100000:1.0:1789760747.161909:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731425713963 time 731482625723 state 0x0 ref 1 00000100:00100000:1.0:1789760747.161923:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495828736:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760747.161974:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23937024, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb807 00010000:00010000:1.0:1789760747.162098:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546798 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.162114:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760747.162118:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760747.162124:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546798 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.162138:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546798 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760747.197740:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23937024 -> 23941120 00010000:00010000:1.0:1789760747.197759:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546798 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.197770:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546798 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.197779:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546798 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.197786:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760747.197790:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546798 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.197799:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546798 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.197816:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546798 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.197829:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546798 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760747.197916:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495828736:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35993us (37809us total) trans 4294973161 rc 0/0 00000100:00100000:1.0:1789760747.197928:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5873 00000100:00100000:1.0:1789760747.197937:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731425713963 time 731482625723 state 0x2 ref 0 00000001:00080000:3.0:1789760747.198821:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973161 is committed 00000100:00100000:3.0:1789760747.203722:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760747.204432:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495828864 00000100:00100000:0.0:1789760747.204442:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495828864 00000100:00100000:0.0:1789760747.204497:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731425713963 time 731482625723 state 0x0 ref 1 00000100:00100000:0.0:1789760747.204511:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731482886139 time 731482625723 state 0x1 ref 1 00000100:00100000:0.0:1789760747.204545:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731482886139 time 731525262365 state 0x0 ref 1 00000100:00100000:0.0:1789760747.204631:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495828864:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760747.204679:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23941120, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb807 00010000:00010000:0.0:1789760747.204688:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454679f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.204702:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760747.204707:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.204712:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454679f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.204727:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454679f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760747.259711:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973162 is committed 00002000:00010000:3.0:1789760747.259830:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23941120 -> 23945216 00010000:00010000:3.0:1789760747.259846:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454679f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.259859:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454679f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.259875:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454679f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.259883:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ae00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760747.259887:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454679f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.259896:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454679f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.259917:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454679f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.259925:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454679f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.260057:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495828864:12345-192.168.203.9@tcp:4:dd.0 Request processed in 55425us (56337us total) trans 4294973162 rc 0/0 00000100:00100000:3.0:1789760747.260077:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5874 00000100:00100000:3.0:1789760747.260087:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731482886139 time 731525262365 state 0x2 ref 0 00000100:00100000:1.0:1789760747.270148:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760747.270218:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495828992 00000100:00100000:0.0:1789760747.270229:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495828992 00000100:00100000:0.0:1789760747.270492:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731482886139 time 731525262365 state 0x0 ref 1 00000100:00100000:0.0:1789760747.270503:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731525522781 time 731525262365 state 0x1 ref 1 00000100:00100000:0.0:1789760747.270562:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731525522781 time 731591279431 state 0x0 ref 1 00000100:00100000:0.0:1789760747.270574:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495828992:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760747.270629:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23945216, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb807 00010000:00010000:0.0:1789760747.270641:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545467a6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.270657:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760747.270662:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.270668:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545467a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.270682:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545467a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760747.324935:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973163 is committed 00002000:00010000:3.0:1789760747.329587:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23945216 -> 23949312 00010000:00010000:3.0:1789760747.329607:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545467a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.329626:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545467a6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.329634:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545467a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.329641:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ae00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760747.329646:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545467a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.329655:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545467a6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.329676:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545467a6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.329683:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545467a6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.329796:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495828992:12345-192.168.203.9@tcp:4:dd.0 Request processed in 59219us (59651us total) trans 4294973163 rc 0/0 00000100:00100000:3.0:1789760747.329806:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5875 00000100:00100000:3.0:1789760747.329817:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731525522781 time 731591279431 state 0x2 ref 0 00000100:00100000:1.0:1789760747.337569:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760747.337650:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495829120 00000100:00100000:2.0:1789760747.337669:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495829120 00000100:00100000:2.0:1789760747.337721:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731525522781 time 731591279431 state 0x0 ref 1 00000100:00100000:2.0:1789760747.337737:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731591539847 time 731591279431 state 0x1 ref 1 00000100:00100000:2.0:1789760747.337767:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731591539847 time 731658483166 state 0x0 ref 1 00000100:00100000:2.0:1789760747.337781:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495829120:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760747.337835:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23949312, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb807 00010000:00010000:2.0:1789760747.337844:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545467ad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.337865:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760747.337870:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.337877:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545467ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.337890:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545467ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.358793:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760747.381685:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973164 is committed 00002000:00010000:3.0:1789760747.382344:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23949312 -> 23953408 00010000:00010000:3.0:1789760747.382361:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545467ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.382373:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545467ad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.382382:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545467ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.382389:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760747.382394:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545467ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.382403:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545467ad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.382424:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545467ad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.382431:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545467ad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.382528:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495829120:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44746us (44971us total) trans 4294973164 rc 0/0 00000100:00100000:3.0:1789760747.382539:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5876 00000100:00100000:3.0:1789760747.382550:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731591539847 time 731658483166 state 0x2 ref 0 00000100:00100000:3.0:1789760747.387504:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760747.387605:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495829248 00000100:00100000:0.0:1789760747.387614:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495829248 00000100:00100000:0.0:1789760747.387670:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731591539847 time 731658483166 state 0x0 ref 1 00000100:00100000:0.0:1789760747.387685:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731658743582 time 731658483166 state 0x1 ref 1 00000100:00100000:0.0:1789760747.387718:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731658743582 time 731708434629 state 0x0 ref 1 00000100:00100000:0.0:1789760747.387730:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495829248:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760747.387777:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23953408, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb807 00010000:00010000:0.0:1789760747.387787:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545467b4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.387802:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760747.387807:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.387813:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545467b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.387825:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545467b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760747.426636:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973165 is committed 00002000:00010000:3.0:1789760747.426737:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23953408 -> 23957504 00010000:00010000:3.0:1789760747.426753:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545467b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.426765:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545467b4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.426774:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545467b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.426781:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760747.426798:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545467b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.426809:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545467b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.426829:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545467b4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.426836:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545467b4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.426930:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495829248:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39198us (39430us total) trans 4294973165 rc 0/0 00000100:00100000:3.0:1789760747.426945:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5877 00000100:00100000:3.0:1789760747.426955:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731658743582 time 731708434629 state 0x2 ref 0 00000100:00100000:1.0:1789760747.433252:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760747.433504:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495829376 00000100:00100000:3.0:1789760747.433522:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495829376 00000100:00100000:3.0:1789760747.433592:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731658743582 time 731708434629 state 0x0 ref 1 00000100:00100000:3.0:1789760747.433606:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731708695045 time 731708434629 state 0x1 ref 1 00000100:00100000:3.0:1789760747.433635:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731708695045 time 731754352160 state 0x0 ref 1 00000100:00100000:3.0:1789760747.433647:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495829376:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760747.433689:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23957504, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb807 00010000:00010000:3.0:1789760747.433698:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545467bb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.433713:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760747.433717:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.433723:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545467bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.433737:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545467bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760747.471274:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23957504 -> 23961600 00010000:00010000:2.0:1789760747.471316:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545467bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.471330:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545467bb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.471338:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545467bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.471345:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760747.471350:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545467bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.471359:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545467bb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.471385:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545467bb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760747.471389:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973166 is committed 00010000:00010000:2.0:1789760747.471392:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545467bb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.471690:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495829376:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38038us (38440us total) trans 4294973166 rc 0/0 00000100:00100000:2.0:1789760747.471700:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5878 00000100:00100000:2.0:1789760747.471713:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731708695045 time 731754352160 state 0x2 ref 0 00000100:00100000:1.0:1789760747.475149:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760747.475330:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495829504 00000100:00100000:3.0:1789760747.475340:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495829504 00000100:00100000:3.0:1789760747.475395:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731708695045 time 731754352160 state 0x0 ref 1 00000100:00100000:3.0:1789760747.475410:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731754612576 time 731754352160 state 0x1 ref 1 00000100:00100000:3.0:1789760747.475438:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731754612576 time 731796155066 state 0x0 ref 1 00000100:00100000:3.0:1789760747.475450:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01c8c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495829504:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760747.475495:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23961600, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb807 00010000:00010000:3.0:1789760747.475504:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545467c2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.475520:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760747.475524:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.475531:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545467c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.475544:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545467c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760747.518403:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973167 is committed 00002000:00010000:3.0:1789760747.518532:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23961600 -> 23965696 00010000:00010000:3.0:1789760747.518547:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545467c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.518623:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545467c2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.518634:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545467c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.518642:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760747.518647:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545467c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.518656:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545467c2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.518675:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545467c2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.518683:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545467c2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.518776:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01c8c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495829504:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43319us (43628us total) trans 4294973167 rc 0/0 00000100:00100000:3.0:1789760747.518787:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5879 00000100:00100000:3.0:1789760747.518798:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731754612576 time 731796155066 state 0x2 ref 0 00000100:00100000:3.0:1789760747.522349:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760747.522622:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495829632 00000100:00100000:0.0:1789760747.522632:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495829632 00000100:00100000:0.0:1789760747.522686:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731754612576 time 731796155066 state 0x0 ref 1 00000100:00100000:0.0:1789760747.522701:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731796415482 time 731796155066 state 0x1 ref 1 00000100:00100000:3.0:1789760747.522909:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731796415482 time 731843623633 state 0x0 ref 1 00000100:00100000:3.0:1789760747.522923:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495829632:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760747.522971:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23965696, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb807 00010000:00010000:3.0:1789760747.523053:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545467c9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.523070:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760747.523075:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.523081:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545467c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.523094:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545467c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760747.573355:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23965696 -> 23969792 00010000:00010000:1.0:1789760747.573375:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545467c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.573388:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545467c9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.573398:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545467c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.573410:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760747.573415:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545467c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.573425:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545467c9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.573445:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545467c9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.573452:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545467c9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760747.573680:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495829632:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50755us (51335us total) trans 4294973168 rc 0/0 00000100:00100000:1.0:1789760747.573692:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5880 00000100:00100000:1.0:1789760747.573701:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731796415482 time 731843623633 state 0x2 ref 0 00000001:00080000:2.0:1789760747.573991:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973168 is committed 00000100:00100000:1.0:1789760747.580107:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760747.580214:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495829760 00000100:00100000:2.0:1789760747.580223:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495829760 00000100:00100000:2.0:1789760747.580276:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731796415482 time 731843623633 state 0x0 ref 1 00000100:00100000:2.0:1789760747.580413:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731843884049 time 731843623633 state 0x1 ref 1 00000100:00100000:2.0:1789760747.580467:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731843884049 time 731901183452 state 0x0 ref 1 00000100:00100000:2.0:1789760747.580555:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01c8d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495829760:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760747.580604:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23969792, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb807 00010000:00010000:2.0:1789760747.580613:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545467d0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.580636:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760747.580641:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.580648:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545467d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.580662:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545467d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760747.613860:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973169 is committed 00002000:00010000:2.0:1789760747.618044:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23969792 -> 23973888 00010000:00010000:2.0:1789760747.618064:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545467d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.618077:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545467d0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.618086:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545467d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.618094:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760747.618159:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545467d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.618170:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545467d0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.618197:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545467d0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.618204:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545467d0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.618591:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01c8d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495829760:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38108us (38489us total) trans 4294973169 rc 0/0 00000100:00100000:2.0:1789760747.618604:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5881 00000100:00100000:2.0:1789760747.618615:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731843884049 time 731901183452 state 0x2 ref 0 00000100:00100000:2.0:1789760747.622458:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760747.622624:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495829888 00000100:00100000:3.0:1789760747.622634:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495829888 00000100:00100000:3.0:1789760747.622684:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731843884049 time 731901183452 state 0x0 ref 1 00000100:00100000:3.0:1789760747.622698:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731901443868 time 731901183452 state 0x1 ref 1 00000100:00100000:3.0:1789760747.622718:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731901443868 time 731943435997 state 0x0 ref 1 00000100:00100000:3.0:1789760747.622729:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495829888:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760747.622774:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23973888, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb807 00010000:00010000:3.0:1789760747.622783:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545467d7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.622799:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760747.622803:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.622809:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545467d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.622828:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545467d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760747.668858:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23973888 -> 23977984 00002000:00010000:3.0:1789760747.668868:0:13059:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760746 -> 1789760747 00002000:00010000:3.0:1789760747.668870:0:13059:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760746 -> 1789760747 00010000:00010000:3.0:1789760747.668885:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545467d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.668899:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545467d7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.668909:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545467d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.668916:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760747.668922:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545467d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.668930:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545467d7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.668950:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545467d7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.668963:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545467d7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.669076:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495829888:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46343us (46621us total) trans 4294973170 rc 0/0 00000100:00100000:3.0:1789760747.669087:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5882 00000100:00100000:3.0:1789760747.669098:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731901443868 time 731943435997 state 0x2 ref 0 00000001:00080000:1.0:1789760747.670165:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973170 is committed 00000100:00100000:2.0:1789760747.672264:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760747.672456:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495830016 00000100:00100000:1.0:1789760747.672466:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495830016 00000100:00100000:1.0:1789760747.672517:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731901443868 time 731943435997 state 0x0 ref 1 00000100:00100000:1.0:1789760747.672532:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731943696413 time 731943435997 state 0x1 ref 1 00000100:00100000:1.0:1789760747.672569:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731943696413 time 731993285071 state 0x0 ref 1 00000100:00100000:1.0:1789760747.672581:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495830016:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760747.672626:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23977984, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb807 00010000:00010000:1.0:1789760747.672635:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545467de lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.672651:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760747.672655:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760747.672661:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545467de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.672674:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545467de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760747.697901:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23977984 -> 23982080 00010000:00010000:2.0:1789760747.697921:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545467de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.697933:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545467de lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.697942:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545467de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.697949:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760747.697954:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545467de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.697963:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545467de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.697992:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545467de lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.698008:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545467de lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.698095:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495830016:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25511us (25834us total) trans 4294973171 rc 0/0 00000100:00100000:2.0:1789760747.698105:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5883 00000100:00100000:2.0:1789760747.698115:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731943696413 time 731993285071 state 0x2 ref 0 00000001:00080000:2.0:1789760747.698602:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973171 is committed 00000100:00100000:2.0:1789760747.704571:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760747.704794:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495830144 00000100:00100000:0.0:1789760747.704804:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495830144 00000100:00100000:0.0:1789760747.704863:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731943696413 time 731993285071 state 0x0 ref 1 00000100:00100000:0.0:1789760747.704879:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731993545487 time 731993285071 state 0x1 ref 1 00000100:00100000:0.0:1789760747.704910:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731993545487 time 732025626773 state 0x0 ref 1 00000100:00100000:0.0:1789760747.704922:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495830144:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760747.704985:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23982080, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb807 00010000:00010000:0.0:1789760747.705003:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545467e5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.705018:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760747.705022:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.705028:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545467e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.705040:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545467e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760747.758505:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23982080 -> 23986176 00010000:00010000:3.0:1789760747.758527:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545467e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.758541:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545467e5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.758551:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545467e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.758559:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760747.758564:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545467e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.758573:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545467e5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760747.758575:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973172 is committed 00010000:00010000:3.0:1789760747.758601:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545467e5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.758609:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545467e5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.758803:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495830144:12345-192.168.203.9@tcp:4:dd.0 Request processed in 53878us (54236us total) trans 4294973172 rc 0/0 00000100:00100000:3.0:1789760747.758814:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5884 00000100:00100000:3.0:1789760747.758824:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731993545487 time 732025626773 state 0x2 ref 0 00000100:00100000:3.0:1789760747.773539:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760747.774334:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495830272 00000100:00100000:2.0:1789760747.774344:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495830272 00000100:00100000:2.0:1789760747.774814:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 731993545487 time 732025626773 state 0x0 ref 1 00000100:00100000:2.0:1789760747.774829:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732025887189 time 732025626773 state 0x1 ref 1 00000100:00100000:2.0:1789760747.774862:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732025887189 time 732095579563 state 0x0 ref 1 00000100:00100000:2.0:1789760747.774875:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495830272:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760747.774921:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23986176, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb807 00010000:00010000:2.0:1789760747.774930:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545467ec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.774946:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760747.774950:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.774956:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545467ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.774970:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545467ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.806529:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f0276a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:59c4e264-9b75-482b-8813-3d1dc124f384:3317:1876699520308096:0@lo:400:kworker.0 00000100:00100000:0.0:1789760747.806700:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760747.806788:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520308096 00000100:00100000:3.0:1789760747.806796:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520308096 00000100:00100000:3.0:1789760747.806838:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 403 00000100:00100000:3.0:1789760747.806846:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929012310450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:59c4e264-9b75-482b-8813-3d1dc124f384+19:3317:x1876699520308096:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760747.807820:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f0276bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3318:1876699520308224:0@lo:400:kworker.0 00000100:00100000:1.0:1789760747.807861:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760747.808778:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929012310450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:59c4e264-9b75-482b-8813-3d1dc124f384+19:3317:x1876699520308096:12345-0@lo:400:kworker.0 Request processed in 1927us (2073us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760747.808790:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 403 00000100:00100000:3.0:1789760747.808838:0:12710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520308224 00000100:00100000:3.0:1789760747.808844:0:12710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520308224 00000100:00100000:3.0:1789760747.808889:0:12710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 487 00000100:00100000:3.0:1789760747.808899:0:12710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f873480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3318:x1876699520308224:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760747.808951:0:12710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f873480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3318:x1876699520308224:12345-0@lo:400:kworker.0 Request processed in 55us (1090us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760747.808961:0:12710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 487 00000100:00100000:3.0:1789760747.809196:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f0276bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3318:1876699520308224:0@lo:400:kworker.0 00000100:00100000:2.0:1789760747.809399:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f0276a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520308480:0@lo:400:kworker.0 00000100:00100000:2.0:1789760747.809437:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760747.809460:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789760747.809466:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f02769180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699520308352:0@lo:400:kworker.0 00000100:00100000:2.0:1789760747.809487:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760747.809511:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520308480 00000100:00100000:2.0:1789760747.809515:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520308480 00000100:00100000:2.0:1789760747.809554:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 226 00000100:00100000:2.0:1789760747.809560:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f923100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520308480:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760747.809674:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520308352 00000100:00100000:1.0:1789760747.809678:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520308352 00000100:00100000:1.0:1789760747.809705:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 488 00000100:00100000:1.0:1789760747.809712:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f922300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699520308352:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760747.809751:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f922300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699520308352:12345-0@lo:400:kworker.0 Request processed in 41us (263us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760747.809757:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 488 00000100:00100000:1.0:1789760747.809784:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f02769180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699520308352:0@lo:400:kworker.0 00000100:00080000:3.0:1789760747.809846:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789760747.809851:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760747.809943:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff92903f921c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3318:1876699520308736:0@lo:400:kworker.0 00000100:00100000:3.0:1789760747.809980:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760747.810009:0:3318:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760747.810015:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f0276bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3318:1876699520308608:0@lo:400:kworker.0 00000100:00100000:3.0:1789760747.810895:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760747.811186:0:8003:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520308608 00000100:00100000:1.0:1789760747.811194:0:8003:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520308608 00000100:00100000:1.0:1789760747.811245:0:8003:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 489 00000100:00100000:1.0:1789760747.811254:0:8003:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1876699520308608:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760747.811319:0:8003:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1876699520308608:12345-0@lo:400:kworker.0 Request processed in 66us (423us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760747.811326:0:8003:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 489 00000100:00100000:1.0:1789760747.811365:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f0276bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3318:1876699520308608:0@lo:400:kworker.0 00000100:00080000:3.0:1789760747.811453:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760747.811533:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f923100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520308480:12345-0@lo:400:kworker.0 Request processed in 1972us (2095us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760747.811540:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 226 00000100:00100000:3.0:1789760747.811554:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520308736 00000100:00100000:3.0:1789760747.811557:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520308736 00000100:00100000:3.0:1789760747.811594:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 227 00000100:00100000:3.0:1789760747.811600:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3318:x1876699520308736:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760747.811634:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3318:x1876699520308736:12345-0@lo:400:kworker.0 Request processed in 34us (1662us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760747.811640:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 227 00000100:00080000:3.0:1789760747.811652:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789760747.811782:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f0276a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:59c4e264-9b75-482b-8813-3d1dc124f384:3317:1876699520308096:0@lo:400:kworker.0 00000100:00100000:3.0:1789760747.815687:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff92903f921c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3318:1876699520308736:0@lo:400:kworker.0 00000100:00100000:1.0:1789760747.815960:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f0276a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520308480:0@lo:400:kworker.0 00000001:00080000:1.0:1789760747.817501:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973173 is committed 00002000:00010000:3.0:1789760747.817718:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23986176 -> 23990272 00010000:00010000:3.0:1789760747.817733:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545467ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.817752:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545467ec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.817761:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545467ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.817769:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760747.817774:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545467ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.817783:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545467ec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.817804:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545467ec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.817812:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545467ec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.817903:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495830272:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43026us (44371us total) trans 4294973173 rc 0/0 00000100:00100000:3.0:1789760747.817914:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5885 00000100:00100000:3.0:1789760747.817924:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732025887189 time 732095579563 state 0x2 ref 0 00000100:00100000:2.0:1789760747.821987:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760747.822130:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495830400 00000100:00100000:1.0:1789760747.822140:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495830400 00000100:00100000:1.0:1789760747.822189:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732025887189 time 732095579563 state 0x0 ref 1 00000100:00100000:1.0:1789760747.822204:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732095839979 time 732095579563 state 0x1 ref 1 00000100:00100000:1.0:1789760747.822233:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732095839979 time 732142950177 state 0x0 ref 1 00000100:00100000:1.0:1789760747.822244:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f921c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495830400:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760747.822310:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23990272, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb807 00010000:00010000:1.0:1789760747.822327:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545467f3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.822343:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760747.822348:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760747.822354:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545467f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.822368:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545467f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760747.897118:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973174 is committed 00002000:00010000:2.0:1789760747.897199:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23990272 -> 23994368 00010000:00010000:2.0:1789760747.897215:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545467f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.897234:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545467f3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.897242:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545467f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.897249:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760747.897254:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545467f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.897263:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545467f3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.897304:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545467f3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.897313:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545467f3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.897406:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f921c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495830400:12345-192.168.203.9@tcp:4:dd.0 Request processed in 75158us (75431us total) trans 4294973174 rc 0/0 00000100:00100000:2.0:1789760747.897417:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5886 00000100:00100000:2.0:1789760747.897428:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732095839979 time 732142950177 state 0x2 ref 0 00000100:00100000:2.0:1789760747.903833:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760747.903926:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495830528 00000100:00100000:3.0:1789760747.903937:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495830528 00000100:00100000:3.0:1789760747.903986:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732095839979 time 732142950177 state 0x0 ref 1 00000100:00100000:3.0:1789760747.904021:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732143210593 time 732142950177 state 0x1 ref 1 00000100:00100000:3.0:1789760747.904053:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732143210593 time 732224769772 state 0x0 ref 1 00000100:00100000:3.0:1789760747.904065:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495830528:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760747.904114:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23994368, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb807 00010000:00010000:3.0:1789760747.904123:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545467fa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.904140:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760747.904144:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.904150:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545467fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.904165:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545467fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760747.963951:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973175 is committed 00002000:00010000:3.0:1789760747.964022:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23994368 -> 23998464 00010000:00010000:3.0:1789760747.964037:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545467fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.964050:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545467fa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.964060:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545467fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.964068:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760747.964073:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545467fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.964082:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545467fa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.964100:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545467fa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.964107:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545467fa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.964197:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495830528:12345-192.168.203.9@tcp:4:dd.0 Request processed in 60128us (60368us total) trans 4294973175 rc 0/0 00000100:00100000:3.0:1789760747.964207:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5887 00000100:00100000:3.0:1789760747.964216:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732143210593 time 732224769772 state 0x2 ref 0 00000100:00100000:1.0:1789760747.969044:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760747.969134:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495830656 00000100:00100000:3.0:1789760747.969144:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495830656 00000100:00100000:3.0:1789760747.969191:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732143210593 time 732224769772 state 0x0 ref 1 00000100:00100000:3.0:1789760747.969206:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732225030188 time 732224769772 state 0x1 ref 1 00000100:00100000:3.0:1789760747.969239:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732225030188 time 732289955322 state 0x0 ref 1 00000100:00100000:3.0:1789760747.969250:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495830656:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760747.969419:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23998464, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb807 00010000:00010000:3.0:1789760747.969428:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546801 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.969445:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760747.969449:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.969455:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546801 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.969468:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546801 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760747.998350:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973176 is committed 00002000:00010000:0.0:1789760747.998504:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23998464 -> 24002560 00010000:00010000:0.0:1789760747.998520:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546801 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.998534:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546801 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.998544:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546801 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.998560:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760747.998566:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546801 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.998576:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546801 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.998595:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546801 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.998604:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546801 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.998689:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495830656:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29435us (29705us total) trans 4294973176 rc 0/0 00000100:00100000:0.0:1789760747.998698:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5888 00000100:00100000:0.0:1789760747.998708:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732225030188 time 732289955322 state 0x2 ref 0 00000100:00100000:3.0:1789760748.001857:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760748.002344:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495830784 00000100:00100000:0.0:1789760748.002354:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495830784 00000100:00100000:0.0:1789760748.002411:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732225030188 time 732289955322 state 0x0 ref 1 00000100:00100000:0.0:1789760748.002425:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732290215738 time 732289955322 state 0x1 ref 1 00000100:00100000:0.0:1789760748.002452:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732290215738 time 732323168993 state 0x0 ref 1 00000100:00100000:0.0:1789760748.002464:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495830784:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760748.002514:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24002560, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb807 00010000:00010000:0.0:1789760748.002524:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546808 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.002540:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760748.002544:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.002726:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546808 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.002742:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546808 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760748.052391:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973177 is committed 00002000:00010000:0.0:1789760748.052533:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24002560 -> 24006656 00010000:00010000:0.0:1789760748.052550:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546808 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.052563:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546808 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.052572:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546808 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.052579:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760748.052584:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546808 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.052593:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546808 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.052619:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546808 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.052627:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546808 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.052750:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495830784:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50283us (50898us total) trans 4294973177 rc 0/0 00000100:00100000:0.0:1789760748.052761:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5889 00000100:00100000:0.0:1789760748.052770:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732290215738 time 732323168993 state 0x2 ref 0 00000100:00100000:3.0:1789760748.060946:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760748.061029:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495830912 00000100:00100000:0.0:1789760748.061039:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495830912 00000100:00100000:0.0:1789760748.061089:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732290215738 time 732323168993 state 0x0 ref 1 00000100:00100000:0.0:1789760748.061103:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732323429409 time 732323168993 state 0x1 ref 1 00000100:00100000:0.0:1789760748.061137:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732323429409 time 732381854503 state 0x0 ref 1 00000100:00100000:0.0:1789760748.061149:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495830912:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760748.061197:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24006656, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb807 00010000:00010000:0.0:1789760748.061205:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454680f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.061220:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760748.061225:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.061232:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454680f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.061246:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454680f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.087420:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973178 is committed 00002000:00010000:3.0:1789760748.087768:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24006656 -> 24010752 00010000:00010000:3.0:1789760748.087784:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454680f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.087797:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454680f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.087812:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454680f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.087819:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760748.087824:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454680f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.087833:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454680f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.087851:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454680f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.087860:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454680f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.087961:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495830912:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26808us (27019us total) trans 4294973178 rc 0/0 00000100:00100000:3.0:1789760748.088550:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5890 00000100:00100000:3.0:1789760748.088560:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732323429409 time 732381854503 state 0x2 ref 0 00000100:00100000:1.0:1789760748.093200:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760748.093329:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495831040 00000100:00100000:0.0:1789760748.093338:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495831040 00000100:00100000:0.0:1789760748.093395:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732323429409 time 732381854503 state 0x0 ref 1 00000100:00100000:0.0:1789760748.093410:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732382114919 time 732381854503 state 0x1 ref 1 00000100:00100000:0.0:1789760748.093431:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732382114919 time 732414147691 state 0x0 ref 1 00000100:00100000:0.0:1789760748.093443:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495831040:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760748.093491:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24010752, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb807 00010000:00010000:0.0:1789760748.093500:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546816 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.093517:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760748.093522:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.093530:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546816 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.093544:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546816 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.132805:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973179 is committed 00002000:00010000:0.0:1789760748.132912:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24010752 -> 24014848 00010000:00010000:0.0:1789760748.132926:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546816 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.132938:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546816 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.132947:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546816 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.132955:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760748.132960:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546816 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.132980:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546816 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.133009:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546816 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.133017:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546816 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.133112:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495831040:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39666us (39917us total) trans 4294973179 rc 0/0 00000100:00100000:0.0:1789760748.133126:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5891 00000100:00100000:0.0:1789760748.133136:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732382114919 time 732414147691 state 0x2 ref 0 00000100:00100000:3.0:1789760748.136397:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760748.136691:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495831168 00000100:00100000:0.0:1789760748.136703:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495831168 00000100:00100000:0.0:1789760748.136760:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732382114919 time 732414147691 state 0x0 ref 1 00000100:00100000:0.0:1789760748.136775:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732414408107 time 732414147691 state 0x1 ref 1 00000100:00100000:0.0:1789760748.136807:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732414408107 time 732457523603 state 0x0 ref 1 00000100:00100000:0.0:1789760748.136820:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495831168:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760748.136864:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24014848, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb807 00010000:00010000:0.0:1789760748.136872:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454681d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.136889:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760748.136893:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.136899:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454681d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.136913:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454681d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760748.170219:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24014848 -> 24018944 00010000:00010000:0.0:1789760748.170239:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454681d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.170252:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454681d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.170261:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454681d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.170269:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760748.170274:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454681d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.170467:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454681d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.170487:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454681d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.170495:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454681d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.170587:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495831168:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33765us (34195us total) trans 4294973180 rc 0/0 00000100:00100000:0.0:1789760748.170597:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5892 00000100:00100000:0.0:1789760748.170607:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732414408107 time 732457523603 state 0x2 ref 0 00000001:00080000:1.0:1789760748.173406:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973180 is committed 00000100:00100000:3.0:1789760748.174155:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760748.174240:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495831296 00000100:00100000:0.0:1789760748.174248:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495831296 00000100:00100000:0.0:1789760748.174333:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732414408107 time 732457523603 state 0x0 ref 1 00000100:00100000:0.0:1789760748.174349:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732457784019 time 732457523603 state 0x1 ref 1 00000100:00100000:0.0:1789760748.174380:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732457784019 time 732495097274 state 0x0 ref 1 00000100:00100000:0.0:1789760748.174392:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495831296:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760748.174440:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24018944, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb807 00010000:00010000:0.0:1789760748.174449:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546824 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.174466:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760748.174471:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.174477:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546824 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.174490:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546824 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.205838:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973181 is committed 00002000:00010000:3.0:1789760748.205941:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24018944 -> 24023040 00010000:00010000:3.0:1789760748.205957:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546824 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.205971:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546824 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.205997:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546824 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.206006:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760748.206011:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546824 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.206021:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546824 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.206041:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546824 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.206049:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546824 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.206139:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495831296:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31743us (31984us total) trans 4294973181 rc 0/0 00000100:00100000:3.0:1789760748.206149:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5893 00000100:00100000:3.0:1789760748.206159:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732457784019 time 732495097274 state 0x2 ref 0 00000100:00100000:3.0:1789760748.211166:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760748.211380:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495831424 00000100:00100000:0.0:1789760748.211392:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495831424 00000100:00100000:0.0:1789760748.211447:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732457784019 time 732495097274 state 0x0 ref 1 00000100:00100000:0.0:1789760748.211462:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732495357690 time 732495097274 state 0x1 ref 1 00000100:00100000:0.0:1789760748.211542:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732495357690 time 732532258083 state 0x0 ref 1 00000100:00100000:0.0:1789760748.211560:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495831424:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760748.211608:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24023040, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb807 00010000:00010000:0.0:1789760748.211618:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454682b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.211649:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760748.211654:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.211660:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454682b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.211675:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454682b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.233960:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973182 is committed 00002000:00010000:3.0:1789760748.234183:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24023040 -> 24027136 00010000:00010000:3.0:1789760748.234199:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454682b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.234344:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454682b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.234353:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454682b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.234362:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760748.234367:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454682b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.234377:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454682b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.234397:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454682b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.234406:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454682b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.234499:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495831424:12345-192.168.203.9@tcp:4:dd.0 Request processed in 22937us (23337us total) trans 4294973182 rc 0/0 00000100:00100000:3.0:1789760748.234509:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5894 00000100:00100000:3.0:1789760748.234519:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732495357690 time 732532258083 state 0x2 ref 0 00000100:00100000:1.0:1789760748.238570:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760748.238735:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495831552 00000100:00100000:0.0:1789760748.238745:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495831552 00000100:00100000:0.0:1789760748.238802:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732495357690 time 732532258083 state 0x0 ref 1 00000100:00100000:0.0:1789760748.238817:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732532518499 time 732532258083 state 0x1 ref 1 00000100:00100000:0.0:1789760748.238840:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732532518499 time 732559557143 state 0x0 ref 1 00000100:00100000:0.0:1789760748.238853:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495831552:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760748.238904:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24027136, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb807 00010000:00010000:0.0:1789760748.238912:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546832 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.238928:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760748.238931:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.238936:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546832 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.238947:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546832 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760748.281457:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24027136 -> 24031232 00010000:00010000:3.0:1789760748.281480:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546832 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.281493:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546832 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.281501:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546832 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.281509:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760748.281513:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546832 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.281521:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546832 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.281540:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546832 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.281548:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546832 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.281728:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495831552:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42870us (43160us total) trans 4294973183 rc 0/0 00000100:00100000:3.0:1789760748.281740:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5895 00000100:00100000:3.0:1789760748.281750:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732532518499 time 732559557143 state 0x2 ref 0 00000001:00080000:0.0:1789760748.282040:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973183 is committed 00000100:00100000:2.0:1789760748.289507:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760748.289835:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495831680 00000100:00100000:3.0:1789760748.289846:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495831680 00000100:00100000:3.0:1789760748.289899:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732532518499 time 732559557143 state 0x0 ref 1 00000100:00100000:3.0:1789760748.289915:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732559817559 time 732559557143 state 0x1 ref 1 00000100:00100000:3.0:1789760748.289949:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732559817559 time 732610665562 state 0x0 ref 1 00000100:00100000:3.0:1789760748.289961:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903639a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495831680:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760748.290018:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24031232, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb807 00010000:00010000:3.0:1789760748.290028:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546839 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.290044:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760748.290048:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.290055:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546839 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.290068:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546839 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.319015:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff92900fab3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520308864:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760748.319085:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760748.319323:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520308864 00000100:00100000:3.0:1789760748.319331:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520308864 00000100:00100000:3.0:1789760748.319382:0:6711:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 202 00000100:00100000:3.0:1789760748.319391:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520308864:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760748.319429:0:6711:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760748.319481:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520308864:12345-0@lo:13:osp-pre-1-0.0 Request processed in 91us (396us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760748.319489:0:6711:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 202 00000100:00100000:3.0:1789760748.319532:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff92900fab3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520308864:0@lo:13:osp-pre-1-0.0 00000001:00080000:0.0:1789760748.333398:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973184 is committed 00002000:00010000:1.0:1789760748.333533:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24031232 -> 24035328 00010000:00010000:1.0:1789760748.333559:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546839 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.333572:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546839 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.333581:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546839 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.333588:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760748.333593:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546839 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.333607:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546839 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.333626:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546839 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.333634:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546839 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.333736:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903639a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495831680:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43771us (44233us total) trans 4294973184 rc 0/0 00000100:00100000:1.0:1789760748.333746:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5896 00000100:00100000:1.0:1789760748.333756:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732559817559 time 732610665562 state 0x2 ref 0 00000100:00100000:1.0:1789760748.337897:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760748.338051:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495831808 00000100:00100000:3.0:1789760748.338061:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495831808 00000100:00100000:3.0:1789760748.338237:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732559817559 time 732610665562 state 0x0 ref 1 00000100:00100000:3.0:1789760748.338253:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732610925978 time 732610665562 state 0x1 ref 1 00000100:00100000:3.0:1789760748.338365:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732610925978 time 732659081085 state 0x0 ref 1 00000100:00100000:3.0:1789760748.338380:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495831808:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760748.338431:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24035328, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb807 00010000:00010000:3.0:1789760748.338441:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546840 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.338457:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760748.338462:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.338469:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546840 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.338484:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546840 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760748.373947:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973185 is committed 00002000:00010000:0.0:1789760748.374080:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24035328 -> 24039424 00010000:00010000:0.0:1789760748.374097:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546840 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.374110:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546840 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.374119:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546840 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.374126:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760748.374130:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546840 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.374147:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546840 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.374168:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546840 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.374176:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546840 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.374847:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495831808:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36458us (36946us total) trans 4294973185 rc 0/0 00000100:00100000:1.0:1789760748.374859:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5897 00000100:00100000:1.0:1789760748.374870:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732610925978 time 732659081085 state 0x2 ref 0 00000100:00100000:2.0:1789760748.378878:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760748.379207:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495831936 00000100:00100000:3.0:1789760748.379218:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495831936 00000100:00100000:3.0:1789760748.379274:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732610925978 time 732659081085 state 0x0 ref 1 00000100:00100000:3.0:1789760748.379411:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732659341501 time 732659081085 state 0x1 ref 1 00000100:00100000:0.0:1789760748.379579:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732659341501 time 732700293509 state 0x0 ref 1 00000100:00100000:0.0:1789760748.379594:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495831936:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760748.379644:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24039424, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb807 00010000:00010000:0.0:1789760748.379653:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546847 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.379669:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760748.379673:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.379679:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546847 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.379693:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546847 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.386728:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760748.416076:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973186 is committed 00002000:00010000:0.0:1789760748.416258:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24039424 -> 24043520 00010000:00010000:0.0:1789760748.416276:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546847 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.416324:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546847 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.416333:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546847 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.416341:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760748.416346:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546847 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.416356:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546847 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.416377:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546847 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.416385:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546847 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.416469:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495831936:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36873us (37594us total) trans 4294973186 rc 0/0 00000100:00100000:0.0:1789760748.416479:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5898 00000100:00100000:0.0:1789760748.416489:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732659341501 time 732700293509 state 0x2 ref 0 00000100:00100000:2.0:1789760748.419617:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760748.419702:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495832064 00000100:00100000:1.0:1789760748.419712:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495832064 00000100:00100000:1.0:1789760748.419765:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732659341501 time 732700293509 state 0x0 ref 1 00000100:00100000:1.0:1789760748.419780:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732700553925 time 732700293509 state 0x1 ref 1 00000100:00100000:1.0:1789760748.419854:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732700553925 time 732740570316 state 0x0 ref 1 00000100:00100000:1.0:1789760748.419867:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495832064:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760748.419915:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24043520, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb807 00010000:00010000:1.0:1789760748.419924:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454684e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.419940:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760748.419945:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.419952:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454684e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.419978:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454684e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760748.448934:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973187 is committed 00002000:00010000:2.0:1789760748.449487:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24043520 -> 24047616 00010000:00010000:2.0:1789760748.449503:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454684e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.449516:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454684e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.449525:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454684e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.449533:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760748.449537:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454684e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.449545:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454684e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.449565:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454684e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.449573:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454684e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.449701:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495832064:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29831us (30088us total) trans 4294973187 rc 0/0 00000100:00100000:2.0:1789760748.449711:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5899 00000100:00100000:2.0:1789760748.449722:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732700553925 time 732740570316 state 0x2 ref 0 00000100:00100000:1.0:1789760748.453927:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760748.454220:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495832192 00000100:00100000:3.0:1789760748.454230:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495832192 00000100:00100000:3.0:1789760748.454464:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732700553925 time 732740570316 state 0x0 ref 1 00000100:00100000:3.0:1789760748.454481:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732740830732 time 732740570316 state 0x1 ref 1 00000100:00100000:3.0:1789760748.454511:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732740830732 time 732775227616 state 0x0 ref 1 00000100:00100000:3.0:1789760748.454523:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495832192:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760748.454570:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24047616, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb807 00010000:00010000:3.0:1789760748.454579:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546855 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.454594:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760748.454599:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.454605:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546855 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.454619:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546855 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760748.483769:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973188 is committed 00002000:00010000:1.0:1789760748.483884:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24047616 -> 24051712 00010000:00010000:1.0:1789760748.483899:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546855 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.483912:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546855 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.483921:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546855 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.483933:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760748.483938:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546855 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.483947:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546855 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.484042:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546855 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.484053:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546855 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.484264:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495832192:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29737us (30339us total) trans 4294973188 rc 0/0 00000100:00100000:1.0:1789760748.484274:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5900 00000100:00100000:1.0:1789760748.484328:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732740830732 time 732775227616 state 0x2 ref 0 00000100:00100000:1.0:1789760748.488430:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760748.488528:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495832320 00000100:00100000:0.0:1789760748.488538:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495832320 00000100:00100000:0.0:1789760748.488592:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732740830732 time 732775227616 state 0x0 ref 1 00000100:00100000:0.0:1789760748.488606:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732775488032 time 732775227616 state 0x1 ref 1 00000100:00100000:0.0:1789760748.488637:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732775488032 time 732809353551 state 0x0 ref 1 00000100:00100000:0.0:1789760748.488649:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495832320:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760748.488696:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24051712, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb807 00010000:00010000:0.0:1789760748.488706:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454685c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.488723:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760748.488727:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.488734:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454685c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.488748:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454685c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.522982:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973189 is committed 00002000:00010000:0.0:1789760748.523413:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24051712 -> 24055808 00010000:00010000:0.0:1789760748.523428:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454685c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.523446:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454685c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.523455:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454685c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.523462:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760748.523467:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454685c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.523475:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454685c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.523494:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454685c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.523502:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454685c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.523586:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495832320:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34934us (35159us total) trans 4294973189 rc 0/0 00000100:00100000:0.0:1789760748.523597:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5901 00000100:00100000:0.0:1789760748.523607:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732775488032 time 732809353551 state 0x2 ref 0 00000100:00100000:2.0:1789760748.535814:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760748.536013:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495832448 00000100:00100000:1.0:1789760748.536023:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495832448 00000100:00100000:1.0:1789760748.536077:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732775488032 time 732809353551 state 0x0 ref 1 00000100:00100000:1.0:1789760748.536467:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732809613967 time 732809353551 state 0x1 ref 1 00000100:00100000:2.0:1789760748.536607:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732809613967 time 732857321360 state 0x0 ref 1 00000100:00100000:2.0:1789760748.536623:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495832448:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760748.536680:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24055808, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb807 00010000:00010000:2.0:1789760748.536689:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546863 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.536707:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760748.536712:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.536718:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546863 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.536733:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546863 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.577771:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973190 is committed 00002000:00010000:3.0:1789760748.577885:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24055808 -> 24059904 00010000:00010000:3.0:1789760748.577901:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546863 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.577914:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546863 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.577924:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546863 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.577932:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760748.577938:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546863 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.577947:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546863 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.578043:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546863 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.578053:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546863 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.578570:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495832448:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41944us (42760us total) trans 4294973190 rc 0/0 00000100:00100000:3.0:1789760748.578585:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5902 00000100:00100000:3.0:1789760748.578595:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732809613967 time 732857321360 state 0x2 ref 0 00000100:00100000:1.0:1789760748.583793:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760748.583912:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495832576 00000100:00100000:3.0:1789760748.583923:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495832576 00000100:00100000:3.0:1789760748.584049:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732809613967 time 732857321360 state 0x0 ref 1 00000100:00100000:3.0:1789760748.584066:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732857581776 time 732857321360 state 0x1 ref 1 00000100:00100000:3.0:1789760748.584207:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732857581776 time 732904921438 state 0x0 ref 1 00000100:00100000:3.0:1789760748.584221:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495832576:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760748.584269:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24059904, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb807 00010000:00010000:3.0:1789760748.584305:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454686a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.584331:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760748.584336:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.584343:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454686a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.584357:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454686a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.619184:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973191 is committed 00002000:00010000:1.0:1789760748.619250:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24059904 -> 24064000 00010000:00010000:1.0:1789760748.619265:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454686a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.619298:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454686a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.619383:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454686a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.619394:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760748.619399:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454686a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.619408:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454686a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.619430:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454686a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.619438:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454686a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.619535:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495832576:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35312us (35747us total) trans 4294973191 rc 0/0 00000100:00100000:1.0:1789760748.619547:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5903 00000100:00100000:1.0:1789760748.619557:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732857581776 time 732904921438 state 0x2 ref 0 00000100:00100000:2.0:1789760748.625844:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760748.625964:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495832704 00000100:00100000:0.0:1789760748.625996:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495832704 00000100:00100000:0.0:1789760748.626053:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732857581776 time 732904921438 state 0x0 ref 1 00000100:00100000:0.0:1789760748.626069:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732905181854 time 732904921438 state 0x1 ref 1 00000100:00100000:0.0:1789760748.626101:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732905181854 time 732946817488 state 0x0 ref 1 00000100:00100000:0.0:1789760748.626114:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495832704:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760748.626160:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24064000, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb807 00010000:00010000:0.0:1789760748.626170:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546871 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.626186:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760748.626190:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.626196:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546871 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.626209:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546871 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760748.673793:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973192 is committed 00002000:00010000:1.0:1789760748.674118:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24064000 -> 24068096 00002000:00010000:1.0:1789760748.674123:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760747 -> 1789760748 00002000:00010000:1.0:1789760748.674125:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760747 -> 1789760748 00010000:00010000:1.0:1789760748.674139:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546871 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.674152:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546871 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.674161:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546871 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.674169:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760748.674174:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546871 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.674183:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546871 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.674202:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546871 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.674208:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546871 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.674490:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495832704:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48373us (48650us total) trans 4294973192 rc 0/0 00000100:00100000:1.0:1789760748.674501:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5904 00000100:00100000:1.0:1789760748.674510:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732905181854 time 732946817488 state 0x2 ref 0 00000100:00100000:2.0:1789760748.680968:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760748.681141:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495832832 00000100:00100000:0.0:1789760748.681152:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495832832 00000100:00100000:0.0:1789760748.681208:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732905181854 time 732946817488 state 0x0 ref 1 00000100:00100000:0.0:1789760748.681224:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732947077904 time 732946817488 state 0x1 ref 1 00000100:00100000:0.0:1789760748.681258:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732947077904 time 733001975241 state 0x0 ref 1 00000100:00100000:0.0:1789760748.681275:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495832832:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760748.682619:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24068096, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb807 00010000:00010000:1.0:1789760748.682631:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546878 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.682647:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760748.682653:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.682659:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546878 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.682674:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546878 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760748.730691:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973193 is committed 00002000:00010000:1.0:1789760748.731208:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24068096 -> 24072192 00010000:00010000:1.0:1789760748.731223:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546878 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.731234:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546878 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.731242:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546878 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.731249:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760748.731254:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546878 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.731262:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546878 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.731313:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546878 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.731321:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546878 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.731415:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495832832:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50137us (50451us total) trans 4294973193 rc 0/0 00000100:00100000:1.0:1789760748.731425:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5905 00000100:00100000:1.0:1789760748.731435:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732947077904 time 733001975241 state 0x2 ref 0 00000100:00100000:3.0:1789760748.743048:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760748.743923:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495832960 00000100:00100000:1.0:1789760748.743940:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495832960 00000100:00100000:1.0:1789760748.744033:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 732947077904 time 733001975241 state 0x0 ref 1 00000100:00100000:1.0:1789760748.744049:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733002235657 time 733001975241 state 0x1 ref 1 00000100:00100000:1.0:1789760748.744087:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733002235657 time 733064804106 state 0x0 ref 1 00000100:00100000:1.0:1789760748.744104:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495832960:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760748.744152:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24072192, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb807 00010000:00010000:1.0:1789760748.744162:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454687f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.744178:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760748.744182:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.744189:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454687f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.744203:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454687f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760748.788834:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973194 is committed 00002000:00010000:1.0:1789760748.789123:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24072192 -> 24076288 00010000:00010000:1.0:1789760748.789139:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454687f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.789151:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454687f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.789161:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454687f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.789168:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760748.789173:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454687f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.789442:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454687f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.789463:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454687f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.789470:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454687f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.789685:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495832960:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45578us (46640us total) trans 4294973194 rc 0/0 00000100:00100000:2.0:1789760748.789697:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5906 00000100:00100000:2.0:1789760748.789707:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733002235657 time 733064804106 state 0x2 ref 0 00000100:00100000:3.0:1789760748.802854:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760748.803114:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495833088 00000100:00100000:1.0:1789760748.803124:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495833088 00000100:00100000:1.0:1789760748.803187:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733002235657 time 733064804106 state 0x0 ref 1 00000100:00100000:1.0:1789760748.803203:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733065064522 time 733064804106 state 0x1 ref 1 00000100:00100000:1.0:1789760748.803801:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733065064522 time 733124514577 state 0x0 ref 1 00000100:00100000:1.0:1789760748.803817:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495833088:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760748.803868:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24076288, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb807 00010000:00010000:1.0:1789760748.803877:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546886 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.803894:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760748.803899:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.803905:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546886 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.803920:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546886 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760748.842192:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973195 is committed 00002000:00010000:3.0:1789760748.842249:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24076288 -> 24080384 00010000:00010000:3.0:1789760748.842273:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546886 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.842306:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546886 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.842312:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546886 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.842317:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760748.842320:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546886 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.842326:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546886 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.842341:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546886 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.842345:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546886 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.842430:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495833088:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38612us (39580us total) trans 4294973195 rc 0/0 00000100:00100000:3.0:1789760748.842440:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5907 00000100:00100000:3.0:1789760748.842450:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733065064522 time 733124514577 state 0x2 ref 0 00000100:00100000:3.0:1789760748.845232:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760748.845484:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495833216 00000100:00100000:1.0:1789760748.845494:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495833216 00000100:00100000:1.0:1789760748.845677:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733065064522 time 733124514577 state 0x0 ref 1 00000100:00100000:1.0:1789760748.845692:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733124774993 time 733124514577 state 0x1 ref 1 00000100:00100000:1.0:1789760748.845727:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733124774993 time 733166443173 state 0x0 ref 1 00000100:00100000:1.0:1789760748.845741:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495833216:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760748.845792:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24080384, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb807 00010000:00010000:1.0:1789760748.845801:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454688d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.845818:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760748.845822:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.845828:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454688d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.845841:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454688d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760748.879591:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973196 is committed 00002000:00010000:2.0:1789760748.879661:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24080384 -> 24084480 00010000:00010000:2.0:1789760748.879674:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454688d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.879694:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454688d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.879704:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454688d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.879712:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760748.879717:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454688d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.879726:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454688d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.879748:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454688d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.879756:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454688d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.879855:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495833216:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34112us (34626us total) trans 4294973196 rc 0/0 00000100:00100000:2.0:1789760748.879866:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5908 00000100:00100000:2.0:1789760748.879876:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733124774993 time 733166443173 state 0x2 ref 0 00000100:00100000:1.0:1789760748.882598:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760748.882806:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495833344 00000100:00100000:2.0:1789760748.882816:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495833344 00000100:00100000:2.0:1789760748.882877:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733124774993 time 733166443173 state 0x0 ref 1 00000100:00100000:2.0:1789760748.882893:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733166703589 time 733166443173 state 0x1 ref 1 00000100:00100000:2.0:1789760748.882928:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733166703589 time 733203642069 state 0x0 ref 1 00000100:00100000:2.0:1789760748.882940:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495833344:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760748.883006:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24084480, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb807 00010000:00010000:2.0:1789760748.883016:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546894 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.883535:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760748.883541:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.883548:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546894 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.883564:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546894 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760748.922772:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973197 is committed 00002000:00010000:1.0:1789760748.923015:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24084480 -> 24088576 00010000:00010000:1.0:1789760748.923031:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546894 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.923045:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546894 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.923054:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546894 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.923061:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760748.923065:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546894 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.923074:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546894 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.923108:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546894 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.923115:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546894 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.923203:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495833344:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40261us (40611us total) trans 4294973197 rc 0/0 00000100:00100000:1.0:1789760748.923214:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5909 00000100:00100000:1.0:1789760748.923223:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733166703589 time 733203642069 state 0x2 ref 0 00000100:00100000:3.0:1789760748.928509:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760748.928598:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495833472 00000100:00100000:1.0:1789760748.928608:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495833472 00000100:00100000:1.0:1789760748.928660:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733166703589 time 733203642069 state 0x0 ref 1 00000100:00100000:1.0:1789760748.928676:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733203902485 time 733203642069 state 0x1 ref 1 00000100:00100000:1.0:1789760748.928696:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733203902485 time 733249413669 state 0x0 ref 1 00000100:00100000:1.0:1789760748.928708:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495833472:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760748.928757:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24088576, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb807 00010000:00010000:1.0:1789760748.928766:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454689b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.928783:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760748.928788:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.928794:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454689b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.928807:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454689b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.966219:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973198 is committed 00002000:00010000:2.0:1789760748.966350:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24088576 -> 24092672 00010000:00010000:2.0:1789760748.966367:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454689b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.966380:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454689b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.966389:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454689b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.966397:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760748.966402:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454689b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.966411:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454689b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.966430:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454689b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.966438:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454689b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.966598:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495833472:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37887us (38154us total) trans 4294973198 rc 0/0 00000100:00100000:2.0:1789760748.966612:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5910 00000100:00100000:2.0:1789760748.966622:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733203902485 time 733249413669 state 0x2 ref 0 00000100:00100000:3.0:1789760748.969886:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760748.970234:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495833600 00000100:00100000:2.0:1789760748.970245:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495833600 00000100:00100000:2.0:1789760748.970380:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733203902485 time 733249413669 state 0x0 ref 1 00000100:00100000:2.0:1789760748.970395:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733249674085 time 733249413669 state 0x1 ref 1 00000100:00100000:2.0:1789760748.970456:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733249674085 time 733291171809 state 0x0 ref 1 00000100:00100000:2.0:1789760748.970469:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495833600:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760748.970516:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24092672, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb807 00010000:00010000:2.0:1789760748.970524:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545468a2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.970540:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760748.970544:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.970550:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545468a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.970565:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545468a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760749.016527:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973199 is committed 00002000:00010000:1.0:1789760749.016592:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24092672 -> 24096768 00010000:00010000:1.0:1789760749.016607:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545468a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.016634:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545468a2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.016644:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545468a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.016651:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760749.016656:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545468a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.016664:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545468a2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.016682:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545468a2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.016689:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545468a2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.016960:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495833600:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46486us (47076us total) trans 4294973199 rc 0/0 00000100:00100000:1.0:1789760749.016972:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5911 00000100:00100000:1.0:1789760749.016982:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733249674085 time 733291171809 state 0x2 ref 0 00000100:00100000:3.0:1789760749.020389:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760749.020484:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495833728 00000100:00100000:1.0:1789760749.020493:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495833728 00000100:00100000:1.0:1789760749.020556:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733249674085 time 733291171809 state 0x0 ref 1 00000100:00100000:1.0:1789760749.020571:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733291432225 time 733291171809 state 0x1 ref 1 00000100:00100000:1.0:1789760749.020588:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733291432225 time 733341306239 state 0x0 ref 1 00000100:00100000:1.0:1789760749.020599:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495833728:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760749.020653:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24096768, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb807 00010000:00010000:1.0:1789760749.020662:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468a9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.020678:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760749.020682:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.020688:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.020700:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760749.021309:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760749.044544:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760749.045727:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760749.045753:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760749.045838:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495834112 00000100:00100000:3.0:1789760749.045848:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495834112 00000100:00100000:3.0:1789760749.045904:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495833984 00000100:00100000:3.0:1789760749.045908:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495833984 00000100:00100000:3.0:1789760749.045939:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 490 00000100:00100000:3.0:1789760749.045950:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03095500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2356:x1876699495834112:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:3.0:1789760749.046049:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2356:x1876699495834112:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 101us (327us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760749.046058:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 490 00000100:00100000:3.0:1789760749.046078:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 491 00000100:00100000:3.0:1789760749.046086:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03095180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:17022965-1e4f-46fb-915c-61730096b8e2+8:2356:x1876699495833984:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:3.0:1789760749.046116:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:17022965-1e4f-46fb-915c-61730096b8e2+8:2356:x1876699495833984:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 31us (361us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760749.046123:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 491 00000100:00080000:3.0:1789760749.046151:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760749.049471:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495833856 00000100:00100000:1.0:1789760749.049487:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495833856 00000100:00100000:1.0:1789760749.049548:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 404 00000100:00100000:1.0:1789760749.049559:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929012310050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2356:x1876699495833856:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:1.0:1789760749.049743:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929012310050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2356:x1876699495833856:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 184us (5207us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760749.049752:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 404 00000001:00080000:0.0:1789760749.058600:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973200 is committed 00002000:00010000:1.0:1789760749.058820:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24096768 -> 24100864 00010000:00010000:1.0:1789760749.058837:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.058850:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468a9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.058860:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.058868:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760749.058873:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.058883:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468a9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.058902:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468a9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.058911:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468a9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.059027:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495833728:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38425us (38642us total) trans 4294973200 rc 0/0 00000100:00100000:1.0:1789760749.059038:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5912 00000100:00100000:1.0:1789760749.059048:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733291432225 time 733341306239 state 0x2 ref 0 00000100:00100000:0.0:1789760749.063639:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760749.064115:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495834240 00000100:00100000:2.0:1789760749.064126:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495834240 00000100:00100000:2.0:1789760749.064182:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733291432225 time 733341306239 state 0x0 ref 1 00000100:00100000:2.0:1789760749.064197:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733341566655 time 733341306239 state 0x1 ref 1 00000100:00100000:2.0:1789760749.064229:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733341566655 time 733384946165 state 0x0 ref 1 00000100:00100000:2.0:1789760749.064242:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495834240:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760749.064336:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24100864, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb807 00010000:00010000:2.0:1789760749.064345:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545468b0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.064362:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760749.064367:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.064373:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545468b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.064388:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545468b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760749.125148:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973201 is committed 00002000:00010000:0.0:1789760749.125428:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24100864 -> 24104960 00010000:00010000:0.0:1789760749.125444:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545468b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.125457:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545468b0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.125473:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545468b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.125481:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760749.125486:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545468b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.125494:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545468b0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.125514:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545468b0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.125525:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545468b0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.125620:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495834240:12345-192.168.203.9@tcp:4:dd.0 Request processed in 61373us (61996us total) trans 4294973201 rc 0/0 00000100:00100000:0.0:1789760749.125631:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5913 00000100:00100000:0.0:1789760749.125641:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733341566655 time 733384946165 state 0x2 ref 0 00000100:00100000:3.0:1789760749.133598:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760749.133703:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495834368 00000100:00100000:1.0:1789760749.133718:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495834368 00000100:00100000:1.0:1789760749.133769:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733341566655 time 733384946165 state 0x0 ref 1 00000100:00100000:1.0:1789760749.133784:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733385206581 time 733384946165 state 0x1 ref 1 00000100:00100000:1.0:1789760749.133819:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733385206581 time 733454535769 state 0x0 ref 1 00000100:00100000:1.0:1789760749.133830:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495834368:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760749.133880:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24104960, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb807 00010000:00010000:1.0:1789760749.133888:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468b7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.133903:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760749.133908:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.133914:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.133926:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760749.166884:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24104960 -> 24109056 00010000:00010000:2.0:1789760749.166902:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.166911:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468b7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.166917:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.166923:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760749.166926:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.166932:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468b7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.166946:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468b7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.166950:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468b7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760749.167051:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973202 is committed 00000100:00100000:2.0:1789760749.167051:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495834368:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33219us (33459us total) trans 4294973202 rc 0/0 00000100:00100000:2.0:1789760749.167061:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5914 00000100:00100000:2.0:1789760749.167070:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733385206581 time 733454535769 state 0x2 ref 0 00000100:00100000:2.0:1789760749.172396:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760749.172636:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495834496 00000100:00100000:1.0:1789760749.172646:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495834496 00000100:00100000:1.0:1789760749.172702:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733385206581 time 733454535769 state 0x0 ref 1 00000100:00100000:1.0:1789760749.172718:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733454796185 time 733454535769 state 0x1 ref 1 00000100:00100000:1.0:1789760749.172789:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733454796185 time 733493506002 state 0x0 ref 1 00000100:00100000:1.0:1789760749.172800:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495834496:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760749.172850:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24109056, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb807 00010000:00010000:1.0:1789760749.172859:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468be lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.172876:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760749.172880:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.172886:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.172900:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760749.200611:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973203 is committed 00002000:00010000:1.0:1789760749.200744:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24109056 -> 24113152 00010000:00010000:1.0:1789760749.200758:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.200770:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468be lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.200779:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.200786:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760749.200792:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.200800:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468be lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.200818:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468be lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.200825:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468be lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.200916:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495834496:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28112us (28632us total) trans 4294973203 rc 0/0 00000100:00100000:1.0:1789760749.200928:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5915 00000100:00100000:1.0:1789760749.200937:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733454796185 time 733493506002 state 0x2 ref 0 00000100:00100000:3.0:1789760749.206846:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760749.206943:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495834624 00000100:00100000:0.0:1789760749.206952:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495834624 00000100:00100000:0.0:1789760749.207021:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733454796185 time 733493506002 state 0x0 ref 1 00000100:00100000:0.0:1789760749.207048:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733493766418 time 733493506002 state 0x1 ref 1 00000100:00100000:0.0:1789760749.207076:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733493766418 time 733527793575 state 0x0 ref 1 00000100:00100000:0.0:1789760749.207088:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495834624:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760749.207133:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24113152, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb807 00010000:00010000:0.0:1789760749.207142:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545468c5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.207158:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760749.207162:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.207168:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545468c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.207182:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545468c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.238097:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973204 is committed 00002000:00010000:0.0:1789760749.238644:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24113152 -> 24117248 00010000:00010000:0.0:1789760749.238659:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545468c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.238671:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545468c5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.238679:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545468c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.238686:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760749.238690:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545468c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.238699:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545468c5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.238716:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545468c5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.238725:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545468c5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.238823:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495834624:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31732us (31982us total) trans 4294973204 rc 0/0 00000100:00100000:0.0:1789760749.238833:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5916 00000100:00100000:0.0:1789760749.238842:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733493766418 time 733527793575 state 0x2 ref 0 00000100:00100000:0.0:1789760749.243718:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760749.243780:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495834752 00000100:00100000:2.0:1789760749.243788:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495834752 00000100:00100000:2.0:1789760749.243832:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733493766418 time 733527793575 state 0x0 ref 1 00000100:00100000:2.0:1789760749.243843:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733528053991 time 733527793575 state 0x1 ref 1 00000100:00100000:2.0:1789760749.243868:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733528053991 time 733564586028 state 0x0 ref 1 00000100:00100000:2.0:1789760749.243877:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495834752:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760749.243912:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24117248, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb807 00010000:00010000:2.0:1789760749.243919:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545468cc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.243932:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760749.243935:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.243939:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545468cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.243949:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545468cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.286186:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973205 is committed 00002000:00010000:1.0:1789760749.287182:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24117248 -> 24121344 00002000:00010000:1.0:1789760749.287188:0:6714:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 47111 -> 49159 00010000:00010000:1.0:1789760749.287201:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545468cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.287213:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545468cc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.287222:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545468cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.287234:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760749.287239:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545468cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.287247:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545468cc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.287264:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545468cc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.287272:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545468cc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.287398:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495834752:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43516us (43684us total) trans 4294973205 rc 0/0 00000100:00100000:1.0:1789760749.287408:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5917 00000100:00100000:1.0:1789760749.287418:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733528053991 time 733564586028 state 0x2 ref 0 00000100:00100000:2.0:1789760749.293364:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760749.295467:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495834880 00000100:00100000:3.0:1789760749.295481:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495834880 00000100:00100000:3.0:1789760749.295539:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733528053991 time 733564586028 state 0x0 ref 1 00000100:00100000:3.0:1789760749.295553:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733564846444 time 733564586028 state 0x1 ref 1 00000100:00100000:3.0:1789760749.296450:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733564846444 time 733617162587 state 0x0 ref 1 00000100:00100000:3.0:1789760749.296468:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495834880:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760749.296518:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24121344, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:3.0:1789760749.296527:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545468d3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.296542:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760749.296546:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.296552:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545468d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.296578:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545468d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.325903:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973206 is committed 00002000:00010000:1.0:1789760749.326208:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24121344 -> 24125440 00010000:00010000:1.0:1789760749.326224:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545468d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.326238:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545468d3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.326251:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545468d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.326259:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760749.326262:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545468d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.326270:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545468d3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.326322:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545468d3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.326330:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545468d3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.326420:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495834880:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29952us (33060us total) trans 4294973206 rc 0/0 00000100:00100000:1.0:1789760749.326429:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5918 00000100:00100000:1.0:1789760749.326439:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733564846444 time 733617162587 state 0x2 ref 0 00000100:00100000:3.0:1789760749.334499:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760749.335043:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495835008 00000100:00100000:0.0:1789760749.335054:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495835008 00000100:00100000:0.0:1789760749.335111:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733564846444 time 733617162587 state 0x0 ref 1 00000100:00100000:0.0:1789760749.335125:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733617423003 time 733617162587 state 0x1 ref 1 00000100:00100000:0.0:1789760749.335159:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733617423003 time 733655875202 state 0x0 ref 1 00000100:00100000:0.0:1789760749.335172:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495835008:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760749.335220:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24125440, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:0.0:1789760749.335230:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545468da lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.335246:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760749.335250:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.335256:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545468da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.335270:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545468da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760749.366947:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973207 is committed 00002000:00010000:0.0:1789760749.367043:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24125440 -> 24129536 00010000:00010000:0.0:1789760749.367059:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545468da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.367072:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545468da lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.367082:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545468da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.367089:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760749.367094:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545468da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.367104:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545468da lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.367123:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545468da lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.367132:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545468da lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.367215:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495835008:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32041us (32722us total) trans 4294973207 rc 0/0 00000100:00100000:0.0:1789760749.367225:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5919 00000100:00100000:0.0:1789760749.367236:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733617423003 time 733655875202 state 0x2 ref 0 00000100:00100000:0.0:1789760749.375775:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760749.375876:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495835136 00000100:00100000:1.0:1789760749.375886:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495835136 00000100:00100000:1.0:1789760749.375938:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733617423003 time 733655875202 state 0x0 ref 1 00000100:00100000:1.0:1789760749.375952:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733656135618 time 733655875202 state 0x1 ref 1 00000100:00100000:1.0:1789760749.376005:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733656135618 time 733696721370 state 0x0 ref 1 00000100:00100000:1.0:1789760749.376018:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495835136:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760749.376065:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24129536, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:1.0:1789760749.376075:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468e1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.376090:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760749.376094:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.376100:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.376114:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.399321:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973208 is committed 00002000:00010000:0.0:1789760749.399414:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24129536 -> 24133632 00010000:00010000:0.0:1789760749.399429:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.399441:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468e1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.399451:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.399458:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760749.399463:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.399472:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468e1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.399491:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468e1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.399499:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545468e1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.399596:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495835136:12345-192.168.203.9@tcp:4:dd.0 Request processed in 23575us (23826us total) trans 4294973208 rc 0/0 00000100:00100000:0.0:1789760749.399606:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5920 00000100:00100000:0.0:1789760749.399616:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733656135618 time 733696721370 state 0x2 ref 0 00000100:00100000:0.0:1789760749.403090:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760749.403171:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495835264 00000100:00100000:1.0:1789760749.403181:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495835264 00000100:00100000:1.0:1789760749.403237:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733656135618 time 733696721370 state 0x0 ref 1 00000100:00100000:1.0:1789760749.403252:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733696981786 time 733696721370 state 0x1 ref 1 00000100:00100000:1.0:1789760749.403358:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733696981786 time 733724073676 state 0x0 ref 1 00000100:00100000:1.0:1789760749.403370:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495835264:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760749.403414:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24133632, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:1.0:1789760749.403423:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545468e8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.403440:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760749.403444:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.403451:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545468e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.403466:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545468e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.407879:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760749.432730:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973209 is committed 00002000:00010000:3.0:1789760749.432792:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24133632 -> 24137728 00010000:00010000:3.0:1789760749.432807:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545468e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.432819:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545468e8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.432828:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545468e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.432835:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760749.432840:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545468e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.432848:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545468e8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.432867:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545468e8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.432874:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545468e8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.432963:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495835264:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29590us (29877us total) trans 4294973209 rc 0/0 00000100:00100000:3.0:1789760749.432974:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5921 00000100:00100000:3.0:1789760749.432997:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733696981786 time 733724073676 state 0x2 ref 0 00000100:00100000:3.0:1789760749.436944:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760749.437059:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495835392 00000100:00100000:0.0:1789760749.437069:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495835392 00000100:00100000:0.0:1789760749.437369:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733696981786 time 733724073676 state 0x0 ref 1 00000100:00100000:0.0:1789760749.437383:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733724334092 time 733724073676 state 0x1 ref 1 00000100:00100000:0.0:1789760749.437407:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733724334092 time 733758123426 state 0x0 ref 1 00000100:00100000:0.0:1789760749.437421:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495835392:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760749.437469:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24137728, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:0.0:1789760749.437477:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545468ef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.437492:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760749.437497:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.437503:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545468ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.437521:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545468ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.464448:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973210 is committed 00002000:00010000:3.0:1789760749.464509:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24137728 -> 24141824 00010000:00010000:3.0:1789760749.464523:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545468ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.464535:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545468ef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.464546:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545468ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.464554:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760749.464558:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545468ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.464567:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545468ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.464587:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545468ef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.464594:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545468ef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.464685:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495835392:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27264us (27746us total) trans 4294973210 rc 0/0 00000100:00100000:3.0:1789760749.464695:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5922 00000100:00100000:3.0:1789760749.464705:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733724334092 time 733758123426 state 0x2 ref 0 00000100:00100000:0.0:1789760749.468407:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760749.468516:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495835520 00000100:00100000:1.0:1789760749.468525:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495835520 00000100:00100000:1.0:1789760749.468576:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733724334092 time 733758123426 state 0x0 ref 1 00000100:00100000:1.0:1789760749.468591:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733758383842 time 733758123426 state 0x1 ref 1 00000100:00100000:1.0:1789760749.468672:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733758383842 time 733789388464 state 0x0 ref 1 00000100:00100000:1.0:1789760749.468685:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495835520:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760749.468736:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24141824, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:1.0:1789760749.468744:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545468f6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.468759:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760749.468764:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.468770:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545468f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.468784:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545468f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760749.511644:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973211 is committed 00002000:00010000:1.0:1789760749.511753:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24141824 -> 24145920 00010000:00010000:1.0:1789760749.511774:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545468f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.511788:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545468f6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.511798:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545468f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.511805:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760749.511809:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545468f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.511819:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545468f6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.511841:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545468f6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.511849:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545468f6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.511955:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495835520:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43266us (43550us total) trans 4294973211 rc 0/0 00000100:00100000:1.0:1789760749.511966:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5923 00000100:00100000:1.0:1789760749.511993:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733758383842 time 733789388464 state 0x2 ref 0 00000100:00100000:2.0:1789760749.518773:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760749.521099:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495835648 00000100:00100000:3.0:1789760749.521120:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495835648 00000100:00100000:3.0:1789760749.521183:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733758383842 time 733789388464 state 0x0 ref 1 00000100:00100000:3.0:1789760749.521198:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733789648880 time 733789388464 state 0x1 ref 1 00000100:00100000:3.0:1789760749.521228:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733789648880 time 733841945278 state 0x0 ref 1 00000100:00100000:3.0:1789760749.521240:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495835648:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760749.521319:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24145920, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:3.0:1789760749.521328:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545468fd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.521344:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760749.521348:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.521354:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545468fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.521367:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545468fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760749.562874:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24145920 -> 24150016 00010000:00010000:1.0:1789760749.562893:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545468fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.562907:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545468fd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.562916:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545468fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.562923:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760749.562927:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545468fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.562936:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545468fd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.562956:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545468fd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.562991:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545468fd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.563085:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495835648:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41841us (44330us total) trans 4294973212 rc 0/0 00000100:00100000:1.0:1789760749.563095:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5924 00000100:00100000:1.0:1789760749.563505:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733789648880 time 733841945278 state 0x2 ref 0 00000001:00080000:2.0:1789760749.563703:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973212 is committed 00000100:00100000:3.0:1789760749.569634:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760749.570094:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495835776 00000100:00100000:0.0:1789760749.570157:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495835776 00000100:00100000:0.0:1789760749.570217:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733789648880 time 733841945278 state 0x0 ref 1 00000100:00100000:0.0:1789760749.570233:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733842205694 time 733841945278 state 0x1 ref 1 00000100:00100000:0.0:1789760749.570369:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733842205694 time 733891085085 state 0x0 ref 1 00000100:00100000:0.0:1789760749.570382:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495835776:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760749.570439:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24150016, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:0.0:1789760749.570449:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546904 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.570465:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760749.570470:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.570476:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546904 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.570491:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546904 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760749.605505:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973213 is committed 00002000:00010000:3.0:1789760749.605529:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24150016 -> 24154112 00010000:00010000:3.0:1789760749.605544:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546904 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.605556:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546904 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.605565:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546904 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.605572:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760749.605577:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546904 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.605585:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546904 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.605615:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546904 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.605623:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546904 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.605724:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495835776:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35339us (36095us total) trans 4294973213 rc 0/0 00000100:00100000:3.0:1789760749.605734:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5925 00000100:00100000:3.0:1789760749.605743:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733842205694 time 733891085085 state 0x2 ref 0 00000100:00100000:0.0:1789760749.609416:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760749.612178:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495835904 00000100:00100000:1.0:1789760749.612192:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495835904 00000100:00100000:1.0:1789760749.612256:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733842205694 time 733891085085 state 0x0 ref 1 00000100:00100000:1.0:1789760749.612272:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733891345501 time 733891085085 state 0x1 ref 1 00000100:00100000:1.0:1789760749.612954:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733891345501 time 733933667829 state 0x0 ref 1 00000100:00100000:1.0:1789760749.613123:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495835904:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760749.613172:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24154112, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:1.0:1789760749.613181:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454690b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.613196:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760749.613201:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.613207:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454690b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.613227:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454690b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760749.646545:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973214 is committed 00002000:00010000:3.0:1789760749.646834:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24154112 -> 24158208 00010000:00010000:3.0:1789760749.646850:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454690b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.646864:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454690b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.646872:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454690b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.646880:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760749.646884:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454690b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.646893:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454690b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.646912:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454690b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.646920:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x493674835454690b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.647025:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495835904:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33914us (37613us total) trans 4294973214 rc 0/0 00000100:00100000:3.0:1789760749.647035:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5926 00000100:00100000:3.0:1789760749.647046:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733891345501 time 733933667829 state 0x2 ref 0 00000100:00100000:2.0:1789760749.651958:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760749.652066:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495836032 00000100:00100000:1.0:1789760749.652076:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495836032 00000100:00100000:1.0:1789760749.652270:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733891345501 time 733933667829 state 0x0 ref 1 00000100:00100000:1.0:1789760749.652367:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733933928245 time 733933667829 state 0x1 ref 1 00000100:00100000:1.0:1789760749.652398:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733933928245 time 733973113735 state 0x0 ref 1 00000100:00100000:1.0:1789760749.652411:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495836032:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760749.652461:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24158208, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:1.0:1789760749.652469:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546912 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.652660:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760749.652665:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.652671:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546912 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.652807:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546912 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760749.685074:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973215 is committed 00002000:00010000:3.0:1789760749.685181:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24158208 -> 24162304 00002000:00010000:3.0:1789760749.685186:0:13059:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760748 -> 1789760749 00002000:00010000:3.0:1789760749.685189:0:13059:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760748 -> 1789760749 00010000:00010000:3.0:1789760749.685202:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546912 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.685214:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546912 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.685222:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546912 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.685228:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760749.685232:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546912 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.685239:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546912 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.685256:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546912 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.685261:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546912 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.685384:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495836032:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32971us (33432us total) trans 4294973215 rc 0/0 00000100:00100000:3.0:1789760749.685394:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5927 00000100:00100000:3.0:1789760749.685404:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733933928245 time 733973113735 state 0x2 ref 0 00000100:00100000:3.0:1789760749.689176:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760749.690917:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495836160 00000100:00100000:0.0:1789760749.690930:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495836160 00000100:00100000:0.0:1789760749.691087:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733933928245 time 733973113735 state 0x0 ref 1 00000100:00100000:0.0:1789760749.691102:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733973374151 time 733973113735 state 0x1 ref 1 00000100:00100000:0.0:1789760749.691166:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733973374151 time 734011882131 state 0x0 ref 1 00000100:00100000:0.0:1789760749.691179:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495836160:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760749.691228:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24162304, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:0.0:1789760749.691237:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546919 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.691258:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760749.691263:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.691269:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546919 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.691315:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546919 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760749.729314:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24162304 -> 24166400 00010000:00010000:0.0:1789760749.729334:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546919 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.729348:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546919 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.729358:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546919 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.729366:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760749.729371:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546919 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.729379:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546919 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.729710:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546919 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.729720:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546919 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.729812:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495836160:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38630us (40641us total) trans 4294973216 rc 0/0 00000100:00100000:0.0:1789760749.729822:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5928 00000100:00100000:0.0:1789760749.729832:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733973374151 time 734011882131 state 0x2 ref 0 00000001:00080000:1.0:1789760749.730345:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973216 is committed 00000100:00100000:1.0:1789760749.733942:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760749.734141:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495836288 00000100:00100000:3.0:1789760749.734151:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495836288 00000100:00100000:3.0:1789760749.734205:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 733973374151 time 734011882131 state 0x0 ref 1 00000100:00100000:3.0:1789760749.734221:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734012142547 time 734011882131 state 0x1 ref 1 00000100:00100000:3.0:1789760749.734330:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734012142547 time 734055045954 state 0x0 ref 1 00000100:00100000:3.0:1789760749.734342:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495836288:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760749.734389:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24166400, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:3.0:1789760749.734398:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546920 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.734414:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760749.734419:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.734426:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546920 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.734439:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546920 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760749.773666:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973217 is committed 00002000:00010000:2.0:1789760749.774148:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24166400 -> 24170496 00010000:00010000:2.0:1789760749.774164:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546920 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.774177:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546920 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.774187:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546920 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.774194:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760749.774199:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546920 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.774208:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546920 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.774228:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546920 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.774236:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546920 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.775517:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495836288:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41166us (41573us total) trans 4294973217 rc 0/0 00000100:00100000:2.0:1789760749.775529:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5929 00000100:00100000:2.0:1789760749.775548:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734012142547 time 734055045954 state 0x2 ref 0 00000100:00100000:3.0:1789760749.778929:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760749.779098:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495836416 00000100:00100000:1.0:1789760749.779109:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495836416 00000100:00100000:1.0:1789760749.779162:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734012142547 time 734055045954 state 0x0 ref 1 00000100:00100000:1.0:1789760749.779177:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734055306370 time 734055045954 state 0x1 ref 1 00000100:00100000:1.0:1789760749.779210:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734055306370 time 734099927373 state 0x0 ref 1 00000100:00100000:1.0:1789760749.779224:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495836416:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760749.779271:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24170496, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:1.0:1789760749.779296:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546927 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.779324:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760749.779332:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.779339:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546927 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.779352:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546927 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.811140:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973218 is committed 00002000:00010000:2.0:1789760749.811251:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24170496 -> 24174592 00010000:00010000:2.0:1789760749.811268:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546927 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.811317:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546927 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.811326:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546927 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.811333:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760749.811337:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546927 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.811347:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546927 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.811367:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546927 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.811376:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546927 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.811468:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495836416:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32242us (32543us total) trans 4294973218 rc 0/0 00000100:00100000:2.0:1789760749.811478:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5930 00000100:00100000:2.0:1789760749.811487:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734055306370 time 734099927373 state 0x2 ref 0 00000100:00100000:3.0:1789760749.816046:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760749.816203:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495836544 00000100:00100000:0.0:1789760749.816213:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495836544 00000100:00100000:0.0:1789760749.816270:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734055306370 time 734099927373 state 0x0 ref 1 00000100:00100000:0.0:1789760749.816327:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734100187789 time 734099927373 state 0x1 ref 1 00000100:00100000:0.0:1789760749.816351:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734100187789 time 734137067699 state 0x0 ref 1 00000100:00100000:0.0:1789760749.816363:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495836544:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760749.816409:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24174592, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:0.0:1789760749.816419:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454692e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.816436:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760749.816441:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.816448:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454692e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.816461:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454692e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.854909:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973219 is committed 00002000:00010000:1.0:1789760749.854916:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24174592 -> 24178688 00010000:00010000:1.0:1789760749.854932:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454692e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.854945:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454692e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.855058:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454692e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.855070:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760749.855076:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454692e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.855086:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454692e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.855151:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454692e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.855160:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454692e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.855249:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495836544:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38883us (39208us total) trans 4294973219 rc 0/0 00000100:00100000:1.0:1789760749.855262:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5931 00000100:00100000:1.0:1789760749.855273:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734100187789 time 734137067699 state 0x2 ref 0 00000100:00100000:3.0:1789760749.859901:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760749.860178:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495836672 00000100:00100000:2.0:1789760749.860189:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495836672 00000100:00100000:2.0:1789760749.860243:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734100187789 time 734137067699 state 0x0 ref 1 00000100:00100000:2.0:1789760749.860259:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734137328115 time 734137067699 state 0x1 ref 1 00000100:00100000:3.0:1789760749.860934:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734137328115 time 734181647012 state 0x0 ref 1 00000100:00100000:3.0:1789760749.860955:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495836672:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760749.861184:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24178688, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:3.0:1789760749.861194:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546935 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.861212:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760749.861217:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.861224:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546935 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.861238:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546935 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760749.903618:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973220 is committed 00002000:00010000:0.0:1789760749.903830:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24178688 -> 24182784 00010000:00010000:0.0:1789760749.903847:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546935 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.904269:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546935 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.904329:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546935 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.904337:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760749.904342:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546935 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.904353:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546935 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.904373:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546935 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.904382:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546935 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.904582:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495836672:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43628us (44686us total) trans 4294973220 rc 0/0 00000100:00100000:0.0:1789760749.904593:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5932 00000100:00100000:0.0:1789760749.904605:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734137328115 time 734181647012 state 0x2 ref 0 00000100:00100000:3.0:1789760749.913953:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760749.914443:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495836800 00000100:00100000:2.0:1789760749.914453:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495836800 00000100:00100000:2.0:1789760749.914511:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734137328115 time 734181647012 state 0x0 ref 1 00000100:00100000:2.0:1789760749.914527:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734181907428 time 734181647012 state 0x1 ref 1 00000100:00100000:2.0:1789760749.914686:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734181907428 time 734235333492 state 0x0 ref 1 00000100:00100000:2.0:1789760749.914703:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495836800:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760749.914751:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24182784, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:2.0:1789760749.914760:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454693c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.914777:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760749.914782:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.914788:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454693c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.914803:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454693c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760749.946946:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973221 is committed 00002000:00010000:3.0:1789760749.947136:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24182784 -> 24186880 00010000:00010000:3.0:1789760749.947152:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454693c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.947172:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454693c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.947182:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454693c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.947190:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760749.947265:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454693c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.947277:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454693c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.947395:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454693c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.947404:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454693c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.947508:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495836800:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32804us (33562us total) trans 4294973221 rc 0/0 00000100:00100000:3.0:1789760749.947519:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5933 00000100:00100000:3.0:1789760749.947529:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734181907428 time 734235333492 state 0x2 ref 0 00000100:00100000:3.0:1789760749.953356:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760749.953539:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495836928 00000100:00100000:1.0:1789760749.953549:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495836928 00000100:00100000:1.0:1789760749.953602:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734181907428 time 734235333492 state 0x0 ref 1 00000100:00100000:1.0:1789760749.953616:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734235593908 time 734235333492 state 0x1 ref 1 00000100:00100000:1.0:1789760749.953654:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734235593908 time 734274371303 state 0x0 ref 1 00000100:00100000:1.0:1789760749.953666:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495836928:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760749.953719:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24186880, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:1.0:1789760749.953729:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546943 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.953747:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760749.953751:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.953757:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546943 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.953771:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546943 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760749.981877:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973222 is committed 00002000:00010000:1.0:1789760749.981947:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24186880 -> 24190976 00010000:00010000:1.0:1789760749.982181:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546943 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.982200:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546943 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.982209:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546943 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.982216:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760749.982220:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546943 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.982230:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546943 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.982251:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546943 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.982260:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546943 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.982574:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495836928:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28904us (29222us total) trans 4294973222 rc 0/0 00000100:00100000:1.0:1789760749.982585:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5934 00000100:00100000:1.0:1789760749.982596:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734235593908 time 734274371303 state 0x2 ref 0 00000100:00100000:1.0:1789760749.986249:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760749.986396:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495837056 00000100:00100000:0.0:1789760749.986406:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495837056 00000100:00100000:0.0:1789760749.986456:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734235593908 time 734274371303 state 0x0 ref 1 00000100:00100000:0.0:1789760749.986475:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734274631719 time 734274371303 state 0x1 ref 1 00000100:00100000:0.0:1789760749.986502:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734274631719 time 734307219551 state 0x0 ref 1 00000100:00100000:0.0:1789760749.986514:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495837056:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760749.986555:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24190976, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:0.0:1789760749.986564:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454694a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.986580:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760749.986584:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.986591:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454694a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.986604:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454694a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760750.022796:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973223 is committed 00002000:00010000:0.0:1789760750.022805:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24190976 -> 24195072 00010000:00010000:0.0:1789760750.022823:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454694a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.022835:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454694a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.022845:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454694a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.022858:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760750.022863:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454694a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.022872:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454694a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.022891:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454694a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.022898:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454694a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.023020:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495837056:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36502us (36774us total) trans 4294973223 rc 0/0 00000100:00100000:0.0:1789760750.023034:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5935 00000100:00100000:0.0:1789760750.023043:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734274631719 time 734307219551 state 0x2 ref 0 00000100:00100000:1.0:1789760750.025702:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760750.025780:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495837184 00000100:00100000:3.0:1789760750.025788:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495837184 00000100:00100000:3.0:1789760750.025836:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734274631719 time 734307219551 state 0x0 ref 1 00000100:00100000:3.0:1789760750.025852:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734307479967 time 734307219551 state 0x1 ref 1 00000100:00100000:3.0:1789760750.025877:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734307479967 time 734346594641 state 0x0 ref 1 00000100:00100000:3.0:1789760750.025890:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495837184:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760750.025935:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24195072, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:3.0:1789760750.025945:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546951 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.025969:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760750.025974:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.025990:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546951 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.026004:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546951 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760750.050657:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973224 is committed 00002000:00010000:1.0:1789760750.050785:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24195072 -> 24199168 00010000:00010000:1.0:1789760750.050802:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546951 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.050816:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546951 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.050826:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546951 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.050833:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760750.050838:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546951 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.050847:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546951 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.050867:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546951 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.050876:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546951 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760750.051096:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495837184:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25203us (25396us total) trans 4294973224 rc 0/0 00000100:00100000:1.0:1789760750.051107:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5936 00000100:00100000:1.0:1789760750.051118:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734307479967 time 734346594641 state 0x2 ref 0 00000100:00100000:3.0:1789760750.057732:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760750.058017:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495837312 00000100:00100000:1.0:1789760750.058028:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495837312 00000100:00100000:1.0:1789760750.058090:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734307479967 time 734346594641 state 0x0 ref 1 00000100:00100000:1.0:1789760750.058106:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734346855057 time 734346594641 state 0x1 ref 1 00000100:00100000:1.0:1789760750.058143:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734346855057 time 734378859121 state 0x0 ref 1 00000100:00100000:1.0:1789760750.058155:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495837312:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760750.058204:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24199168, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:1.0:1789760750.058213:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546958 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.058228:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760750.058232:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760750.058239:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546958 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.058252:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546958 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760750.088225:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973225 is committed 00002000:00010000:1.0:1789760750.088380:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24199168 -> 24203264 00010000:00010000:1.0:1789760750.088396:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546958 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.088415:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546958 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.088425:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546958 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.088434:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760750.088439:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546958 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.088448:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546958 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.088466:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546958 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.088474:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546958 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760750.088605:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495837312:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30447us (30879us total) trans 4294973225 rc 0/0 00000100:00100000:1.0:1789760750.088616:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5937 00000100:00100000:1.0:1789760750.088626:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734346855057 time 734378859121 state 0x2 ref 0 00000100:00100000:2.0:1789760750.092104:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760750.092205:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495837440 00000100:00100000:0.0:1789760750.092214:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495837440 00000100:00100000:0.0:1789760750.092269:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734346855057 time 734378859121 state 0x0 ref 1 00000100:00100000:0.0:1789760750.092333:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734379119537 time 734378859121 state 0x1 ref 1 00000100:00100000:0.0:1789760750.092354:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734379119537 time 734413071113 state 0x0 ref 1 00000100:00100000:0.0:1789760750.092366:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01c8c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495837440:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760750.092414:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24203264, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:0.0:1789760750.092423:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454695f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.092439:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760750.092443:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760750.092449:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454695f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.092468:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454695f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760750.125788:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973226 is committed 00002000:00010000:0.0:1789760750.125874:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24203264 -> 24207360 00010000:00010000:0.0:1789760750.125889:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454695f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.125900:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454695f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.125909:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454695f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.125916:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760750.125920:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454695f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.125929:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454695f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.125948:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454695f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.126027:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454695f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.126159:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01c8c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495837440:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33790us (34061us total) trans 4294973226 rc 0/0 00000100:00100000:0.0:1789760750.126170:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5938 00000100:00100000:0.0:1789760750.126180:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734379119537 time 734413071113 state 0x2 ref 0 00000100:00100000:0.0:1789760750.135623:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760750.135721:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495837568 00000100:00100000:1.0:1789760750.135731:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495837568 00000100:00100000:1.0:1789760750.135783:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734379119537 time 734413071113 state 0x0 ref 1 00000100:00100000:1.0:1789760750.135798:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734413331529 time 734413071113 state 0x1 ref 1 00000100:00100000:1.0:1789760750.135829:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734413331529 time 734456545541 state 0x0 ref 1 00000100:00100000:1.0:1789760750.135840:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495837568:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760750.135887:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24207360, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:1.0:1789760750.135896:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546966 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.135911:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760750.135916:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760750.135922:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546966 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.135935:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546966 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760750.202267:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973227 is committed 00002000:00010000:3.0:1789760750.202372:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24207360 -> 24211456 00010000:00010000:3.0:1789760750.202389:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546966 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.202407:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546966 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.202418:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546966 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.202425:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760750.202429:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546966 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.202438:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546966 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.202458:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546966 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.202465:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546966 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760750.202655:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495837568:12345-192.168.203.9@tcp:4:dd.0 Request processed in 66810us (67035us total) trans 4294973227 rc 0/0 00000100:00100000:1.0:1789760750.202666:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5939 00000100:00100000:1.0:1789760750.202676:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734413331529 time 734456545541 state 0x2 ref 0 00000100:00100000:3.0:1789760750.206017:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760750.206074:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495837696 00000100:00100000:1.0:1789760750.206084:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495837696 00000100:00100000:1.0:1789760750.206137:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734413331529 time 734456545541 state 0x0 ref 1 00000100:00100000:1.0:1789760750.206152:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734456805957 time 734456545541 state 0x1 ref 1 00000100:00100000:1.0:1789760750.206186:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734456805957 time 734526903181 state 0x0 ref 1 00000100:00100000:1.0:1789760750.206199:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f921c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495837696:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760750.206246:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24211456, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:1.0:1789760750.206401:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454696d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.206419:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760750.206424:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760750.206430:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454696d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.206443:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454696d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760750.251331:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24211456 -> 24215552 00010000:00010000:3.0:1789760750.251351:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454696d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.251363:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454696d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.251373:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454696d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.251381:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760750.251386:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454696d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.251394:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454696d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.251415:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454696d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.251422:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454696d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.251506:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f921c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495837696:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45303us (45490us total) trans 4294973228 rc 0/0 00000100:00100000:3.0:1789760750.251516:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5940 00000100:00100000:3.0:1789760750.251527:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734456805957 time 734526903181 state 0x2 ref 0 00000001:00080000:0.0:1789760750.251687:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973228 is committed 00000100:00100000:2.0:1789760750.255662:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760750.255753:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495837824 00000100:00100000:0.0:1789760750.255764:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495837824 00000100:00100000:0.0:1789760750.255821:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734456805957 time 734526903181 state 0x0 ref 1 00000100:00100000:0.0:1789760750.255838:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734527163597 time 734526903181 state 0x1 ref 1 00000100:00100000:0.0:1789760750.255867:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734527163597 time 734576584287 state 0x0 ref 1 00000100:00100000:0.0:1789760750.255879:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01c8d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495837824:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760750.255934:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24215552, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:0.0:1789760750.255944:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546974 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.256052:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760750.256057:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760750.256064:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546974 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.256150:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546974 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760750.315615:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24215552 -> 24219648 00010000:00010000:3.0:1789760750.315637:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546974 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.315656:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546974 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.315665:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546974 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.315673:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760750.315677:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546974 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.315688:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546974 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.315708:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546974 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.315716:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546974 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.315806:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01c8d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495837824:12345-192.168.203.9@tcp:4:dd.0 Request processed in 59924us (60149us total) trans 4294973229 rc 0/0 00000100:00100000:3.0:1789760750.315817:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5941 00000100:00100000:3.0:1789760750.315827:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734527163597 time 734576584287 state 0x2 ref 0 00000001:00080000:1.0:1789760750.315903:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973229 is committed 00000100:00100000:2.0:1789760750.323945:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760750.324029:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495837952 00000100:00100000:3.0:1789760750.324039:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495837952 00000100:00100000:3.0:1789760750.324102:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734527163597 time 734576584287 state 0x0 ref 1 00000100:00100000:3.0:1789760750.324117:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734576844703 time 734576584287 state 0x1 ref 1 00000100:00100000:3.0:1789760750.324146:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734576844703 time 734644862804 state 0x0 ref 1 00000100:00100000:3.0:1789760750.324158:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495837952:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760750.324206:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24219648, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:3.0:1789760750.324215:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454697b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.324230:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760750.324234:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.324240:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454697b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.324253:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454697b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760750.356442:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973230 is committed 00002000:00010000:2.0:1789760750.357244:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24219648 -> 24223744 00010000:00010000:2.0:1789760750.357388:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454697b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.357405:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454697b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.357414:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454697b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.357421:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760750.357426:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454697b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.357434:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454697b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.357456:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454697b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.357462:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454697b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.357619:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495837952:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33457us (33676us total) trans 4294973230 rc 0/0 00000100:00100000:2.0:1789760750.357631:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5942 00000100:00100000:2.0:1789760750.357642:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734576844703 time 734644862804 state 0x2 ref 0 00000100:00100000:2.0:1789760750.362219:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760750.362585:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495838080 00000100:00100000:3.0:1789760750.362594:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495838080 00000100:00100000:3.0:1789760750.362649:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734576844703 time 734644862804 state 0x0 ref 1 00000100:00100000:3.0:1789760750.362669:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734645123220 time 734644862804 state 0x1 ref 1 00000100:00100000:3.0:1789760750.362689:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734645123220 time 734683407138 state 0x0 ref 1 00000100:00100000:3.0:1789760750.362700:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495838080:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760750.362739:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24223744, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:3.0:1789760750.362745:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546982 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.362760:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760750.362763:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.362768:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546982 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.362778:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546982 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760750.397866:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973231 is committed 00002000:00010000:3.0:1789760750.398039:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24223744 -> 24227840 00010000:00010000:3.0:1789760750.398055:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546982 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.398068:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546982 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.398149:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546982 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.398156:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760750.398161:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546982 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.398170:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546982 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.398189:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546982 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.398196:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546982 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.398472:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495838080:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35766us (36257us total) trans 4294973231 rc 0/0 00000100:00100000:3.0:1789760750.398483:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5943 00000100:00100000:3.0:1789760750.398494:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734645123220 time 734683407138 state 0x2 ref 0 00000100:00100000:1.0:1789760750.402845:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760750.402883:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495838208 00000100:00100000:3.0:1789760750.402892:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495838208 00000100:00100000:3.0:1789760750.402942:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734645123220 time 734683407138 state 0x0 ref 1 00000100:00100000:3.0:1789760750.402997:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734683667554 time 734683407138 state 0x1 ref 1 00000100:00100000:3.0:1789760750.403030:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734683667554 time 734723746385 state 0x0 ref 1 00000100:00100000:3.0:1789760750.403218:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495838208:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760750.403362:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24227840, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:3.0:1789760750.403370:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546989 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.403387:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760750.403391:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.403398:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546989 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.403417:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546989 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.433322:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760750.455917:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973232 is committed 00002000:00010000:3.0:1789760750.456169:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24227840 -> 24231936 00010000:00010000:3.0:1789760750.456185:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546989 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.456197:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546989 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.456206:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546989 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.456213:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760750.456217:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546989 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.456226:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546989 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.456245:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546989 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.456253:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546989 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.456463:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495838208:12345-192.168.203.9@tcp:4:dd.0 Request processed in 53244us (53620us total) trans 4294973232 rc 0/0 00000100:00100000:3.0:1789760750.456474:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5944 00000100:00100000:3.0:1789760750.456483:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734683667554 time 734723746385 state 0x2 ref 0 00000100:00100000:2.0:1789760750.460753:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760750.460898:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495838336 00000100:00100000:3.0:1789760750.460908:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495838336 00000100:00100000:3.0:1789760750.460971:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734683667554 time 734723746385 state 0x0 ref 1 00000100:00100000:3.0:1789760750.460995:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734724006801 time 734723746385 state 0x1 ref 1 00000100:00100000:3.0:1789760750.461023:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734724006801 time 734781740191 state 0x0 ref 1 00000100:00100000:3.0:1789760750.461036:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495838336:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760750.461096:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24231936, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:3.0:1789760750.461105:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546990 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.461121:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760750.461125:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.461131:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546990 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.461150:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546990 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760750.492623:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973233 is committed 00002000:00010000:3.0:1789760750.492835:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24231936 -> 24236032 00010000:00010000:3.0:1789760750.492850:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546990 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.492864:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546990 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.492873:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546990 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.492881:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760750.492886:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546990 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.492895:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546990 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.492915:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546990 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.492923:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546990 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.493192:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495838336:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32152us (32443us total) trans 4294973233 rc 0/0 00000100:00100000:3.0:1789760750.493203:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5945 00000100:00100000:3.0:1789760750.493214:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734724006801 time 734781740191 state 0x2 ref 0 00000100:00100000:2.0:1789760750.498220:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760750.498413:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495838464 00000100:00100000:3.0:1789760750.498424:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495838464 00000100:00100000:3.0:1789760750.498479:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734724006801 time 734781740191 state 0x0 ref 1 00000100:00100000:3.0:1789760750.498492:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734782000607 time 734781740191 state 0x1 ref 1 00000100:00100000:3.0:1789760750.498558:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734782000607 time 734819273875 state 0x0 ref 1 00000100:00100000:3.0:1789760750.498570:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495838464:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760750.498617:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24236032, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:3.0:1789760750.498626:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546997 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.498640:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760750.498644:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.498650:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546997 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.498662:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546997 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760750.524543:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973234 is committed 00002000:00010000:3.0:1789760750.524646:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24236032 -> 24240128 00010000:00010000:3.0:1789760750.524661:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546997 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.524685:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546997 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.524694:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546997 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.524701:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760750.524706:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546997 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.524717:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546997 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.524737:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546997 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.524743:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546997 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.524828:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495838464:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26256us (26614us total) trans 4294973234 rc 0/0 00000100:00100000:3.0:1789760750.524841:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5946 00000100:00100000:3.0:1789760750.524851:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734782000607 time 734819273875 state 0x2 ref 0 00000100:00100000:1.0:1789760750.529463:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760750.529539:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495838592 00000100:00100000:3.0:1789760750.529549:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495838592 00000100:00100000:3.0:1789760750.529604:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734782000607 time 734819273875 state 0x0 ref 1 00000100:00100000:3.0:1789760750.529619:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734819534291 time 734819273875 state 0x1 ref 1 00000100:00100000:3.0:1789760750.529648:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734819534291 time 734850365511 state 0x0 ref 1 00000100:00100000:3.0:1789760750.529659:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495838592:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760750.529716:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24240128, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:3.0:1789760750.529725:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454699e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.529739:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760750.529744:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.529750:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454699e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.529764:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454699e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760750.601752:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973235 is committed 00002000:00010000:3.0:1789760750.601904:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24240128 -> 24244224 00010000:00010000:3.0:1789760750.601920:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454699e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.601933:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454699e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.601943:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454699e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.601960:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760750.601966:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454699e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.601984:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454699e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.602004:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454699e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.602012:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454699e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.602101:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495838592:12345-192.168.203.9@tcp:4:dd.0 Request processed in 72438us (72642us total) trans 4294973235 rc 0/0 00000100:00100000:3.0:1789760750.602111:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5947 00000100:00100000:3.0:1789760750.602120:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734819534291 time 734850365511 state 0x2 ref 0 00000100:00100000:2.0:1789760750.606712:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760750.606903:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495838720 00000100:00100000:3.0:1789760750.606913:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495838720 00000100:00100000:3.0:1789760750.607042:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734819534291 time 734850365511 state 0x0 ref 1 00000100:00100000:3.0:1789760750.607058:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734850625927 time 734850365511 state 0x1 ref 1 00000100:00100000:3.0:1789760750.607124:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734850625927 time 734927840422 state 0x0 ref 1 00000100:00100000:3.0:1789760750.607138:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495838720:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760750.607189:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24244224, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:3.0:1789760750.607198:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469a5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.607213:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760750.607217:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.607223:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.607236:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760750.653554:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973236 is committed 00002000:00010000:3.0:1789760750.653889:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24244224 -> 24248320 00010000:00010000:3.0:1789760750.653905:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.653917:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469a5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.653931:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.653939:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760750.653944:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.654393:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469a5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.654416:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469a5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.654424:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469a5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.654561:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495838720:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47421us (47853us total) trans 4294973236 rc 0/0 00000100:00100000:3.0:1789760750.654570:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5948 00000100:00100000:3.0:1789760750.654581:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734850625927 time 734927840422 state 0x2 ref 0 00000100:00100000:0.0:1789760750.659111:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760750.659203:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495838848 00000100:00100000:3.0:1789760750.659213:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495838848 00000100:00100000:3.0:1789760750.659268:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734850625927 time 734927840422 state 0x0 ref 1 00000100:00100000:3.0:1789760750.659371:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734928100838 time 734927840422 state 0x1 ref 1 00000100:00100000:3.0:1789760750.659449:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734928100838 time 734980164990 state 0x0 ref 1 00000100:00100000:3.0:1789760750.659462:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495838848:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760750.659511:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24248320, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:3.0:1789760750.659519:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545469ac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.659535:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760750.659539:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.659545:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545469ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.659558:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545469ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760750.698352:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973237 is committed 00002000:00010000:0.0:1789760750.698489:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24248320 -> 24252416 00002000:00010000:0.0:1789760750.698493:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760749 -> 1789760750 00002000:00010000:0.0:1789760750.698496:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760749 -> 1789760750 00010000:00010000:0.0:1789760750.698509:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545469ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.698522:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545469ac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.698602:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545469ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.698615:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760750.698621:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545469ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.698630:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545469ac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.698652:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545469ac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.698660:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545469ac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.698782:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495838848:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39317us (39675us total) trans 4294973237 rc 0/0 00000100:00100000:0.0:1789760750.698793:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5949 00000100:00100000:0.0:1789760750.698803:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734928100838 time 734980164990 state 0x2 ref 0 00000100:00100000:1.0:1789760750.705480:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760750.705731:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495838976 00000100:00100000:3.0:1789760750.705742:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495838976 00000100:00100000:3.0:1789760750.705800:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734928100838 time 734980164990 state 0x0 ref 1 00000100:00100000:3.0:1789760750.705816:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734980425406 time 734980164990 state 0x1 ref 1 00000100:00100000:3.0:1789760750.705849:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734980425406 time 735026565662 state 0x0 ref 1 00000100:00100000:3.0:1789760750.705860:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495838976:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760750.705921:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24252416, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:3.0:1789760750.705932:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469b3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.705948:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760750.707435:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.707445:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.707465:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760750.751431:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973238 is committed 00002000:00010000:3.0:1789760750.752008:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24252416 -> 24256512 00010000:00010000:3.0:1789760750.752027:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.752040:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469b3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.752048:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.752056:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760750.752061:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.752070:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469b3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.752098:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469b3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.752105:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469b3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.752205:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495838976:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46341us (46731us total) trans 4294973238 rc 0/0 00000100:00100000:3.0:1789760750.752216:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5950 00000100:00100000:3.0:1789760750.752225:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734980425406 time 735026565662 state 0x2 ref 0 00000100:00100000:2.0:1789760750.755854:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760750.756177:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495839104 00000100:00100000:3.0:1789760750.756187:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495839104 00000100:00100000:3.0:1789760750.756243:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 734980425406 time 735026565662 state 0x0 ref 1 00000100:00100000:3.0:1789760750.756260:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735026826078 time 735026565662 state 0x1 ref 1 00000100:00100000:3.0:1789760750.756363:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735026826078 time 735077078948 state 0x0 ref 1 00000100:00100000:3.0:1789760750.756378:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495839104:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760750.756429:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24256512, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:3.0:1789760750.756438:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469ba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.756454:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760750.756460:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.756466:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.756478:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760750.791385:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973239 is committed 00002000:00010000:3.0:1789760750.791704:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24256512 -> 24260608 00010000:00010000:3.0:1789760750.791720:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.791732:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469ba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.791741:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.791749:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760750.791754:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.791763:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469ba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.791784:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469ba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.791797:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469ba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.792080:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495839104:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35699us (36229us total) trans 4294973239 rc 0/0 00000100:00100000:3.0:1789760750.792090:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5951 00000100:00100000:3.0:1789760750.792100:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735026826078 time 735077078948 state 0x2 ref 0 00000100:00100000:2.0:1789760750.802084:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760750.802157:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495839232 00000100:00100000:0.0:1789760750.802167:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495839232 00000100:00100000:0.0:1789760750.802218:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735026826078 time 735077078948 state 0x0 ref 1 00000100:00100000:0.0:1789760750.802235:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735077339364 time 735077078948 state 0x1 ref 1 00000100:00100000:0.0:1789760750.802260:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735077339364 time 735122977623 state 0x0 ref 1 00000100:00100000:0.0:1789760750.802272:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495839232:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760750.802355:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24260608, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:0.0:1789760750.802365:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545469c1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.802381:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760750.802385:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760750.802392:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545469c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.802405:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545469c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760750.864691:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24260608 -> 24264704 00010000:00010000:0.0:1789760750.864711:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545469c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.864724:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545469c1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.864734:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545469c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.864741:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94aa00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760750.864746:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545469c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.864755:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545469c1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.864775:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545469c1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.864783:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545469c1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.864878:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495839232:12345-192.168.203.9@tcp:4:dd.0 Request processed in 62602us (62798us total) trans 4294973240 rc 0/0 00000100:00100000:0.0:1789760750.864889:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5952 00000100:00100000:0.0:1789760750.864899:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735077339364 time 735122977623 state 0x2 ref 0 00000001:00080000:3.0:1789760750.864952:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973240 is committed 00000100:00100000:1.0:1789760750.868379:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760750.868826:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495839360 00000100:00100000:3.0:1789760750.868837:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495839360 00000100:00100000:3.0:1789760750.868892:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735077339364 time 735122977623 state 0x0 ref 1 00000100:00100000:3.0:1789760750.868906:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735123238039 time 735122977623 state 0x1 ref 1 00000100:00100000:3.0:1789760750.868937:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735123238039 time 735189653966 state 0x0 ref 1 00000100:00100000:3.0:1789760750.868959:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903639b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495839360:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760750.869016:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24264704, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:3.0:1789760750.869027:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469c8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.869045:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760750.869049:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.869056:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.869070:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.879461:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f01836680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520308992:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760750.879544:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760750.880492:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520308992 00000100:00100000:2.0:1789760750.880502:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520308992 00000100:00100000:2.0:1789760750.880738:0:6711:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 203 00000100:00100000:2.0:1789760750.880749:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520308992:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760750.880773:0:6711:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527630336 pending 1703936 free 3833593856 avail 3828350976 00000100:00080000:1.0:1789760750.881050:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760750.885707:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520308992:12345-0@lo:13:osp-pre-0-0.0 Request processed in 4950us (6152us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760750.885721:0:6711:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 203 00000100:00100000:3.0:1789760750.890653:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f01836680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520308992:0@lo:13:osp-pre-0-0.0 00000001:00080000:3.0:1789760750.901992:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973241 is committed 00002000:00010000:3.0:1789760750.902051:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24264704 -> 24268800 00010000:00010000:3.0:1789760750.902065:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.902077:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469c8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.902085:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.902094:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760750.902098:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.902108:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.902127:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469c8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.902135:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469c8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.902224:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903639b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495839360:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33272us (33849us total) trans 4294973241 rc 0/0 00000100:00100000:3.0:1789760750.902235:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5953 00000100:00100000:3.0:1789760750.902247:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735123238039 time 735189653966 state 0x2 ref 0 00000100:00100000:1.0:1789760750.907903:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760750.907997:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495839488 00000100:00100000:3.0:1789760750.908007:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495839488 00000100:00100000:3.0:1789760750.908059:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735123238039 time 735189653966 state 0x0 ref 1 00000100:00100000:3.0:1789760750.908075:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735189914382 time 735189653966 state 0x1 ref 1 00000100:00100000:3.0:1789760750.908153:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735189914382 time 735228869597 state 0x0 ref 1 00000100:00100000:3.0:1789760750.908167:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495839488:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760750.908215:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24268800, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:3.0:1789760750.908224:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469cf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.908241:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760750.908246:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.908252:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.908271:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760750.942667:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973242 is committed 00002000:00010000:3.0:1789760750.942752:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24268800 -> 24272896 00010000:00010000:3.0:1789760750.942768:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.942781:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469cf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.942790:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.942797:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760750.942802:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.942811:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469cf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.942830:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469cf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.942838:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469cf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.943092:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495839488:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34922us (35193us total) trans 4294973242 rc 0/0 00000100:00100000:3.0:1789760750.943103:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5954 00000100:00100000:3.0:1789760750.943113:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735189914382 time 735228869597 state 0x2 ref 0 00000100:00100000:0.0:1789760750.952730:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760750.953201:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495839616 00000100:00100000:3.0:1789760750.953212:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495839616 00000100:00100000:3.0:1789760750.953268:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735189914382 time 735228869597 state 0x0 ref 1 00000100:00100000:3.0:1789760750.953303:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735229130013 time 735228869597 state 0x1 ref 1 00000100:00100000:3.0:1789760750.953334:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735229130013 time 735274050992 state 0x0 ref 1 00000100:00100000:3.0:1789760750.953347:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495839616:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760750.953394:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24272896, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:3.0:1789760750.953403:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469d6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.953420:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760750.953424:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.953431:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.953446:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760750.996568:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973243 is committed 00002000:00010000:0.0:1789760750.996668:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24272896 -> 24276992 00010000:00010000:0.0:1789760750.996684:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.996696:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469d6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.996716:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.996728:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760750.996734:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.996743:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469d6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.996762:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469d6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.996831:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545469d6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.996921:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495839616:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43570us (44194us total) trans 4294973243 rc 0/0 00000100:00100000:0.0:1789760750.996932:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5955 00000100:00100000:0.0:1789760750.996942:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735229130013 time 735274050992 state 0x2 ref 0 00000100:00100000:1.0:1789760751.003122:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760751.003200:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495839744 00000100:00100000:3.0:1789760751.003210:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495839744 00000100:00080000:1.0:1789760751.003264:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760751.003312:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735229130013 time 735274050992 state 0x0 ref 1 00000100:00100000:3.0:1789760751.003327:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735274311408 time 735274050992 state 0x1 ref 1 00000100:00100000:3.0:1789760751.003355:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735274311408 time 735324071505 state 0x0 ref 1 00000100:00100000:3.0:1789760751.003367:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04292a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495839744:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760751.003414:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24276992, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:3.0:1789760751.003423:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469dd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.003437:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760751.003441:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.003447:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.003460:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760751.041312:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973244 is committed 00002000:00010000:0.0:1789760751.041415:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24276992 -> 24281088 00010000:00010000:0.0:1789760751.041430:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.041443:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469dd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.041457:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.041464:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760751.041468:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.041476:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469dd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.041495:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469dd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.041503:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545469dd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760751.041590:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04292a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495839744:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38222us (38474us total) trans 4294973244 rc 0/0 00000100:00100000:0.0:1789760751.041600:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5956 00000100:00100000:0.0:1789760751.041610:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735274311408 time 735324071505 state 0x2 ref 0 00000100:00100000:1.0:1789760751.048913:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760751.049092:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495839872 00000100:00100000:3.0:1789760751.049104:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495839872 00000100:00100000:3.0:1789760751.049175:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735274311408 time 735324071505 state 0x0 ref 1 00000100:00100000:3.0:1789760751.049195:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735324331921 time 735324071505 state 0x1 ref 1 00000100:00100000:3.0:1789760751.049227:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735324331921 time 735369943462 state 0x0 ref 1 00000100:00100000:3.0:1789760751.049239:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04290700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495839872:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760751.049317:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24281088, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:3.0:1789760751.049326:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545469e4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.049341:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760751.049354:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.049362:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545469e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.049377:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545469e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760751.131199:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973245 is committed 00002000:00010000:0.0:1789760751.131435:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24281088 -> 24285184 00010000:00010000:0.0:1789760751.131449:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545469e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.131467:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545469e4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.131475:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545469e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.131482:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760751.131487:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545469e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.131495:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545469e4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.131514:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545469e4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.131521:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545469e4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760751.131616:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04290700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495839872:12345-192.168.203.9@tcp:4:dd.0 Request processed in 82375us (82710us total) trans 4294973245 rc 0/0 00000100:00100000:0.0:1789760751.131626:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5957 00000100:00100000:0.0:1789760751.131636:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735324331921 time 735369943462 state 0x2 ref 0 00000100:00100000:3.0:1789760751.140137:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760751.140258:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495840000 00000100:00100000:0.0:1789760751.140268:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495840000 00000100:00100000:1.0:1789760751.143850:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735324331921 time 735369943462 state 0x0 ref 1 00000100:00100000:1.0:1789760751.143870:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735370203878 time 735369943462 state 0x1 ref 1 00000100:00100000:1.0:1789760751.143982:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735370203878 time 735464698541 state 0x0 ref 1 00000100:00100000:1.0:1789760751.144002:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495840000:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760751.144052:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24285184, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:1.0:1789760751.144061:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545469eb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.144078:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760751.144083:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760751.144089:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545469eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.144103:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545469eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760751.205491:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973246 is committed 00002000:00010000:0.0:1789760751.205767:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24285184 -> 24289280 00010000:00010000:0.0:1789760751.205786:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545469eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.205802:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545469eb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.205811:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545469eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.205819:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760751.205824:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545469eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.205832:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545469eb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.205853:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545469eb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.205860:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545469eb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760751.205964:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495840000:12345-192.168.203.9@tcp:4:dd.0 Request processed in 61965us (65829us total) trans 4294973246 rc 0/0 00000100:00100000:0.0:1789760751.206119:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5958 00000100:00100000:0.0:1789760751.206131:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735370203878 time 735464698541 state 0x2 ref 0 00000100:00100000:3.0:1789760751.209964:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760751.210111:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495840128 00000100:00100000:0.0:1789760751.210122:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495840128 00000100:00100000:0.0:1789760751.210174:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735370203878 time 735464698541 state 0x0 ref 1 00000100:00100000:0.0:1789760751.210189:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735464958957 time 735464698541 state 0x1 ref 1 00000100:00100000:0.0:1789760751.210223:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735464958957 time 735530939122 state 0x0 ref 1 00000100:00100000:0.0:1789760751.210236:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495840128:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760751.210331:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24289280, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:0.0:1789760751.210340:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545469f2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.210357:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760751.210361:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760751.210368:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545469f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.210380:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545469f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760751.283525:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973247 is committed 00002000:00010000:0.0:1789760751.283660:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24289280 -> 24293376 00010000:00010000:0.0:1789760751.283675:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545469f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.283694:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545469f2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.283702:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545469f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.283710:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ae00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760751.283715:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545469f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.283725:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545469f2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.283743:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545469f2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.283750:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545469f2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760751.283843:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495840128:12345-192.168.203.9@tcp:4:dd.0 Request processed in 73603us (73894us total) trans 4294973247 rc 0/0 00000100:00100000:0.0:1789760751.283852:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5959 00000100:00100000:0.0:1789760751.283862:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735464958957 time 735530939122 state 0x2 ref 0 00000100:00100000:1.0:1789760751.288146:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760751.288230:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495840256 00000100:00100000:0.0:1789760751.288239:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495840256 00000100:00100000:0.0:1789760751.288336:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735464958957 time 735530939122 state 0x0 ref 1 00000100:00100000:0.0:1789760751.288352:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735531199538 time 735530939122 state 0x1 ref 1 00000100:00100000:0.0:1789760751.288383:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735531199538 time 735609100517 state 0x0 ref 1 00000100:00100000:0.0:1789760751.288395:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495840256:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760751.288441:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24293376, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:0.0:1789760751.288450:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545469f9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.288466:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760751.288471:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760751.288477:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545469f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.288489:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545469f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760751.391546:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24293376 -> 24297472 00010000:00010000:3.0:1789760751.391566:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545469f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.391585:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545469f9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.391593:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545469f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.391600:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94aa00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760751.391605:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545469f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.391614:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545469f9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.391634:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545469f9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.391642:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545469f9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.391734:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495840256:12345-192.168.203.9@tcp:4:dd.0 Request processed in 103336us (103593us total) trans 4294973248 rc 0/0 00000100:00100000:3.0:1789760751.391744:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5960 00000100:00100000:3.0:1789760751.391753:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735531199538 time 735609100517 state 0x2 ref 0 00000001:00080000:3.0:1789760751.391790:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973248 is committed 00000100:00100000:3.0:1789760751.400385:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760751.400586:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495840384 00000100:00100000:0.0:1789760751.400598:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495840384 00000100:00100000:0.0:1789760751.400666:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735531199538 time 735609100517 state 0x0 ref 1 00000100:00100000:0.0:1789760751.400683:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735609360933 time 735609100517 state 0x1 ref 1 00000100:00100000:0.0:1789760751.400719:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735609360933 time 735721434171 state 0x0 ref 1 00000100:00100000:0.0:1789760751.400736:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495840384:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760751.400792:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24297472, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:0.0:1789760751.400803:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546a00 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.400818:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760751.400823:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760751.400830:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546a00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.400844:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546a00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760751.447875:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973249 is committed 00002000:00010000:1.0:1789760751.447894:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24297472 -> 24301568 00010000:00010000:1.0:1789760751.447910:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546a00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.447923:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546a00 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.447930:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546a00 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.447935:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ae00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760751.447939:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546a00 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.447945:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546a00 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.448013:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546a00 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.448024:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546a00 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.448260:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495840384:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47523us (47879us total) trans 4294973249 rc 0/0 00000100:00100000:1.0:1789760751.448271:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5961 00000100:00100000:1.0:1789760751.448322:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735609360933 time 735721434171 state 0x2 ref 0 00000100:00100000:1.0:1789760751.451880:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760751.452077:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495840512 00000100:00100000:0.0:1789760751.452088:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495840512 00000100:00100000:0.0:1789760751.452140:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735609360933 time 735721434171 state 0x0 ref 1 00000100:00100000:0.0:1789760751.452156:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735721694587 time 735721434171 state 0x1 ref 1 00000100:00100000:0.0:1789760751.452188:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735721694587 time 735772904514 state 0x0 ref 1 00000100:00100000:0.0:1789760751.452200:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92901237ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495840512:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760751.452248:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24301568, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:0.0:1789760751.452257:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546a07 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.452575:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760751.452581:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760751.452589:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546a07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.452603:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546a07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.454413:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760751.488652:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973250 is committed 00002000:00010000:0.0:1789760751.488826:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24301568 -> 24305664 00010000:00010000:0.0:1789760751.488843:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546a07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.488856:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546a07 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.488865:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546a07 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.488873:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760751.488878:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546a07 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.488887:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546a07 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.488906:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546a07 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.488914:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546a07 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760751.489015:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92901237ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495840512:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36811us (37138us total) trans 4294973250 rc 0/0 00000100:00100000:0.0:1789760751.489026:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5962 00000100:00100000:0.0:1789760751.489037:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735721694587 time 735772904514 state 0x2 ref 0 00000100:00100000:0.0:1789760751.495152:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760751.495234:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495840640 00000100:00100000:3.0:1789760751.495244:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495840640 00000100:00100000:3.0:1789760751.495346:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735721694587 time 735772904514 state 0x0 ref 1 00000100:00100000:3.0:1789760751.495361:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735773164930 time 735772904514 state 0x1 ref 1 00000100:00100000:3.0:1789760751.495387:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735773164930 time 735816104150 state 0x0 ref 1 00000100:00100000:3.0:1789760751.495400:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495840640:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760751.495445:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24305664, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:3.0:1789760751.495455:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a0e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.495473:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760751.495477:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.495488:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.495502:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760751.529367:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973251 is committed 00002000:00010000:3.0:1789760751.529605:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24305664 -> 24309760 00010000:00010000:3.0:1789760751.529620:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.529638:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a0e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.529648:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a0e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.529655:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760751.529661:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a0e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.529669:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a0e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.529688:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a0e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.529696:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a0e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.529789:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495840640:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34387us (34641us total) trans 4294973251 rc 0/0 00000100:00100000:3.0:1789760751.529798:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5963 00000100:00100000:3.0:1789760751.529807:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735773164930 time 735816104150 state 0x2 ref 0 00000100:00100000:2.0:1789760751.533904:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760751.534043:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495840768 00000100:00100000:1.0:1789760751.534052:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495840768 00000100:00100000:1.0:1789760751.539112:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735773164930 time 735816104150 state 0x0 ref 1 00000100:00100000:1.0:1789760751.539132:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735816364566 time 735816104150 state 0x1 ref 1 00000100:00100000:1.0:1789760751.539160:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735816364566 time 735859877826 state 0x0 ref 1 00000100:00100000:1.0:1789760751.539173:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495840768:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760751.539224:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24309760, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:1.0:1789760751.539234:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a15 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.539250:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760751.539254:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760751.539260:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.539273:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760751.581639:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973252 is committed 00002000:00010000:1.0:1789760751.582497:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24309760 -> 24313856 00010000:00010000:1.0:1789760751.582514:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.582533:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a15 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.582542:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.582548:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760751.582553:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.582562:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a15 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.582584:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a15 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.582592:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a15 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.582688:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495840768:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43502us (48778us total) trans 4294973252 rc 0/0 00000100:00100000:1.0:1789760751.582699:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5964 00000100:00100000:1.0:1789760751.582709:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735816364566 time 735859877826 state 0x2 ref 0 00000100:00100000:1.0:1789760751.591168:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760751.595816:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495840896 00000100:00100000:0.0:1789760751.595832:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495840896 00000100:00100000:1.0:1789760751.596039:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735816364566 time 735859877826 state 0x0 ref 1 00000100:00100000:1.0:1789760751.596055:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735860138242 time 735859877826 state 0x1 ref 1 00000100:00100000:1.0:1789760751.596147:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735860138242 time 735916862420 state 0x0 ref 1 00000100:00100000:1.0:1789760751.596161:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495840896:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760751.596209:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24313856, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:1.0:1789760751.596218:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a1c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.596234:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760751.596239:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760751.596246:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.596261:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760751.644487:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973253 is committed 00002000:00010000:1.0:1789760751.644592:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24313856 -> 24317952 00010000:00010000:1.0:1789760751.644607:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.644618:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a1c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.644626:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a1c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.644635:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760751.644640:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a1c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.644649:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a1c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.644668:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a1c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.644690:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a1c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.644781:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495840896:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48617us (53617us total) trans 4294973253 rc 0/0 00000100:00100000:1.0:1789760751.644792:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5965 00000100:00100000:1.0:1789760751.644802:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735860138242 time 735916862420 state 0x2 ref 0 00000100:00100000:0.0:1789760751.654137:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760751.654227:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495841024 00000100:00100000:3.0:1789760751.654238:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495841024 00000100:00100000:3.0:1789760751.654326:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735860138242 time 735916862420 state 0x0 ref 1 00000100:00100000:3.0:1789760751.654341:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735917122836 time 735916862420 state 0x1 ref 1 00000100:00100000:3.0:1789760751.654371:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735917122836 time 735975087378 state 0x0 ref 1 00000100:00100000:3.0:1789760751.654383:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495841024:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760751.654429:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24317952, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:3.0:1789760751.654438:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a23 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.654455:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760751.654459:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.654466:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.654479:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760751.680833:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973254 is committed 00002000:00010000:1.0:1789760751.681075:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24317952 -> 24322048 00002000:00010000:1.0:1789760751.681080:0:13059:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760750 -> 1789760751 00002000:00010000:1.0:1789760751.681083:0:13059:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760750 -> 1789760751 00010000:00010000:1.0:1789760751.681096:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.681108:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a23 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.681118:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a23 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.681125:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760751.681129:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a23 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.681139:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a23 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.681160:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a23 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.681168:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a23 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.681342:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495841024:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26954us (27208us total) trans 4294973254 rc 0/0 00000100:00100000:1.0:1789760751.681353:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5966 00000100:00100000:1.0:1789760751.681363:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735917122836 time 735975087378 state 0x2 ref 0 00000100:00100000:2.0:1789760751.684323:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760751.684559:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495841152 00000100:00100000:1.0:1789760751.684569:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495841152 00000100:00100000:1.0:1789760751.684784:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735917122836 time 735975087378 state 0x0 ref 1 00000100:00100000:1.0:1789760751.684799:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735975347794 time 735975087378 state 0x1 ref 1 00000100:00100000:1.0:1789760751.684833:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735975347794 time 736005549786 state 0x0 ref 1 00000100:00100000:1.0:1789760751.684845:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495841152:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760751.684895:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24322048, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:1.0:1789760751.684904:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a2a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.684920:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760751.684924:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760751.684930:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.684943:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760751.711730:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973255 is committed 00002000:00010000:2.0:1789760751.711982:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24322048 -> 24326144 00010000:00010000:2.0:1789760751.711998:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.712009:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a2a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.712017:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a2a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.712024:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760751.712029:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a2a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.712038:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a2a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.712057:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a2a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.712065:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546a2a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760751.712244:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495841152:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27396us (27968us total) trans 4294973255 rc 0/0 00000100:00100000:2.0:1789760751.712254:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5967 00000100:00100000:2.0:1789760751.712266:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735975347794 time 736005549786 state 0x2 ref 0 00000100:00100000:3.0:1789760751.715853:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760751.716036:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495841280 00000100:00100000:1.0:1789760751.716045:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495841280 00000100:00100000:1.0:1789760751.716100:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 735975347794 time 736005549786 state 0x0 ref 1 00000100:00100000:1.0:1789760751.716116:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736005810202 time 736005549786 state 0x1 ref 1 00000100:00100000:1.0:1789760751.716148:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736005810202 time 736036864791 state 0x0 ref 1 00000100:00100000:1.0:1789760751.716162:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495841280:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760751.716207:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24326144, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:1.0:1789760751.716216:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546a31 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.716228:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760751.716231:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760751.716235:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546a31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.716245:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546a31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760751.744878:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973256 is committed 00002000:00010000:2.0:1789760751.745059:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24326144 -> 24330240 00010000:00010000:2.0:1789760751.745074:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546a31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.745085:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546a31 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.745093:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546a31 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.745100:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760751.745104:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546a31 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.745113:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546a31 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.745132:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546a31 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.745139:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546a31 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760751.745421:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495841280:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29256us (29571us total) trans 4294973256 rc 0/0 00000100:00100000:2.0:1789760751.745436:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5968 00000100:00100000:2.0:1789760751.745445:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736005810202 time 736036864791 state 0x2 ref 0 00000100:00100000:1.0:1789760751.751619:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760751.752062:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495841408 00000100:00100000:3.0:1789760751.752073:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495841408 00000100:00100000:3.0:1789760751.752131:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736005810202 time 736036864791 state 0x0 ref 1 00000100:00100000:3.0:1789760751.752146:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736037125207 time 736036864791 state 0x1 ref 1 00000100:00100000:3.0:1789760751.752182:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736037125207 time 736072899035 state 0x0 ref 1 00000100:00100000:3.0:1789760751.752194:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495841408:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760751.752241:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24330240, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:3.0:1789760751.752250:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a38 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.752264:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760751.752268:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.752275:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.752407:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760751.791781:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973257 is committed 00002000:00010000:3.0:1789760751.791962:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24330240 -> 24334336 00010000:00010000:3.0:1789760751.791985:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.791998:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a38 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.792008:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a38 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.792016:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760751.792022:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a38 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.792032:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a38 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.792049:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a38 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.792057:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a38 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.792143:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495841408:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39947us (40528us total) trans 4294973257 rc 0/0 00000100:00100000:3.0:1789760751.792153:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5969 00000100:00100000:3.0:1789760751.792163:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736037125207 time 736072899035 state 0x2 ref 0 00000100:00100000:1.0:1789760751.798275:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760751.798403:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495841536 00000100:00100000:2.0:1789760751.798412:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495841536 00000100:00100000:2.0:1789760751.798464:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736037125207 time 736072899035 state 0x0 ref 1 00000100:00100000:2.0:1789760751.798478:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736073159451 time 736072899035 state 0x1 ref 1 00000100:00100000:2.0:1789760751.798500:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736073159451 time 736119216584 state 0x0 ref 1 00000100:00100000:2.0:1789760751.798511:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495841536:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760751.798556:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24334336, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:2.0:1789760751.798565:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546a3f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.798581:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760751.798585:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760751.798591:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546a3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.798604:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546a3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760751.835456:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24334336 -> 24338432 00010000:00010000:3.0:1789760751.835475:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546a3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.835487:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546a3f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.835495:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546a3f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.835508:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760751.835513:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546a3f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.835521:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546a3f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.835538:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546a3f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.835546:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546a3f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.835645:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495841536:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37131us (37374us total) trans 4294973258 rc 0/0 00000100:00100000:3.0:1789760751.835655:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5970 00000100:00100000:3.0:1789760751.835665:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736073159451 time 736119216584 state 0x2 ref 0 00000001:00080000:2.0:1789760751.836533:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973258 is committed 00000100:00100000:1.0:1789760751.841494:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760751.841585:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495841664 00000100:00100000:2.0:1789760751.841594:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495841664 00000100:00100000:2.0:1789760751.841649:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736073159451 time 736119216584 state 0x0 ref 1 00000100:00100000:2.0:1789760751.841664:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736119477000 time 736119216584 state 0x1 ref 1 00000100:00100000:2.0:1789760751.841697:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736119477000 time 736162412825 state 0x0 ref 1 00000100:00100000:2.0:1789760751.841710:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495841664:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760751.841768:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24338432, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:2.0:1789760751.841777:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546a46 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.841793:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760751.841798:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760751.841804:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546a46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.841818:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546a46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760751.886389:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24338432 -> 24342528 00010000:00010000:3.0:1789760751.886409:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546a46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.886422:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546a46 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.886435:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546a46 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.886441:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760751.886446:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546a46 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.886454:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546a46 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.886473:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546a46 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.886480:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546a46 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.886576:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495841664:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44864us (45087us total) trans 4294973259 rc 0/0 00000100:00100000:3.0:1789760751.886586:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5971 00000100:00100000:3.0:1789760751.886597:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736119477000 time 736162412825 state 0x2 ref 0 00000001:00080000:1.0:1789760751.887098:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973259 is committed 00000100:00100000:0.0:1789760751.890167:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760751.890434:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495841792 00000100:00100000:3.0:1789760751.890445:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495841792 00000100:00100000:3.0:1789760751.890496:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736119477000 time 736162412825 state 0x0 ref 1 00000100:00100000:3.0:1789760751.890510:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736162673241 time 736162412825 state 0x1 ref 1 00000100:00100000:3.0:1789760751.890529:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736162673241 time 736211247472 state 0x0 ref 1 00000100:00100000:3.0:1789760751.890541:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495841792:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760751.890744:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24342528, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:3.0:1789760751.890832:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a4d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.890849:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760751.890854:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.890861:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.890876:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760751.916449:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973260 is committed 00002000:00010000:2.0:1789760751.916598:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24342528 -> 24346624 00010000:00010000:2.0:1789760751.916614:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.916626:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a4d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.916639:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a4d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.916646:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760751.916651:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a4d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.916659:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a4d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.916679:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a4d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.916687:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a4d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760751.916824:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495841792:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26279us (26659us total) trans 4294973260 rc 0/0 00000100:00100000:2.0:1789760751.916834:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5972 00000100:00100000:2.0:1789760751.916844:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736162673241 time 736211247472 state 0x2 ref 0 00000100:00100000:3.0:1789760751.930769:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760751.934268:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495841920 00000100:00100000:2.0:1789760751.934616:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495841920 00000100:00100000:0.0:1789760751.938147:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736162673241 time 736211247472 state 0x0 ref 1 00000100:00100000:0.0:1789760751.938168:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736211507888 time 736211247472 state 0x1 ref 1 00000100:00100000:0.0:1789760751.938258:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736211507888 time 736258973871 state 0x0 ref 1 00000100:00100000:0.0:1789760751.938275:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495841920:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760751.938363:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24346624, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:0.0:1789760751.938373:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546a54 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.938390:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760751.938396:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760751.938403:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546a54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.938417:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546a54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760751.982686:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973261 is committed 00002000:00010000:2.0:1789760751.982784:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24346624 -> 24350720 00010000:00010000:2.0:1789760751.982800:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546a54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.982812:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546a54 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.982820:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546a54 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.982827:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760751.982831:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546a54 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.982840:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546a54 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.982859:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546a54 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.982867:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546a54 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760751.983022:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495841920:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44748us (52257us total) trans 4294973261 rc 0/0 00000100:00100000:2.0:1789760751.983032:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5973 00000100:00100000:2.0:1789760751.983041:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736211507888 time 736258973871 state 0x2 ref 0 00000100:00100000:1.0:1789760751.992482:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760751.992566:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495842048 00000100:00100000:3.0:1789760751.992575:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495842048 00000100:00100000:3.0:1789760751.992630:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736211507888 time 736258973871 state 0x0 ref 1 00000100:00100000:3.0:1789760751.992645:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736259234287 time 736258973871 state 0x1 ref 1 00000100:00100000:3.0:1789760751.992666:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736259234287 time 736313382809 state 0x0 ref 1 00000100:00100000:3.0:1789760751.992677:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495842048:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760751.992725:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24350720, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:3.0:1789760751.992734:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546a5b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.992750:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760751.992755:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.992761:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546a5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.992776:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546a5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760752.021711:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973262 is committed 00002000:00010000:1.0:1789760752.021796:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24350720 -> 24354816 00010000:00010000:1.0:1789760752.021813:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546a5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.021826:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546a5b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.021835:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546a5b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.021848:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760752.021854:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546a5b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.021862:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546a5b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.021882:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546a5b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.021890:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546a5b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.022128:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495842048:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29447us (29650us total) trans 4294973262 rc 0/0 00000100:00100000:1.0:1789760752.022139:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5974 00000100:00100000:1.0:1789760752.022148:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736259234287 time 736313382809 state 0x2 ref 0 00000100:00100000:1.0:1789760752.034724:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760752.036370:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495842176 00000100:00100000:2.0:1789760752.036384:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495842176 00000100:00100000:2.0:1789760752.036458:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736259234287 time 736313382809 state 0x0 ref 1 00000100:00100000:2.0:1789760752.036475:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736313643225 time 736313382809 state 0x1 ref 1 00000100:00100000:2.0:1789760752.036508:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736313643225 time 736357225057 state 0x0 ref 1 00000100:00100000:2.0:1789760752.036521:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495842176:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760752.036580:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24354816, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:2.0:1789760752.036590:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546a62 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.036607:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760752.036612:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760752.036618:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546a62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.036633:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546a62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760752.061952:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24354816 -> 24358912 00010000:00010000:2.0:1789760752.061980:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546a62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.061993:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546a62 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.062008:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546a62 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.062015:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760752.062019:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546a62 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.062029:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546a62 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.062048:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546a62 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.062055:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546a62 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760752.062146:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495842176:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25622us (27425us total) trans 4294973263 rc 0/0 00000100:00100000:2.0:1789760752.062156:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5975 00000100:00100000:2.0:1789760752.062165:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736313643225 time 736357225057 state 0x2 ref 0 00000001:00080000:0.0:1789760752.066409:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973263 is committed 00000100:00100000:3.0:1789760752.067736:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760752.067809:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495842304 00000100:00100000:2.0:1789760752.067819:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495842304 00000100:00100000:2.0:1789760752.067873:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736313643225 time 736357225057 state 0x0 ref 1 00000100:00100000:2.0:1789760752.067888:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736357485473 time 736357225057 state 0x1 ref 1 00000100:00100000:2.0:1789760752.067919:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736357485473 time 736388635519 state 0x0 ref 1 00000100:00100000:2.0:1789760752.067932:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495842304:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760752.067986:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24358912, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:2.0:1789760752.067994:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546a69 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.068009:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760752.068013:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760752.068019:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546a69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.068032:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546a69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760752.100729:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973264 is committed 00002000:00010000:0.0:1789760752.100873:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24358912 -> 24363008 00010000:00010000:0.0:1789760752.100889:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546a69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.100900:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546a69 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.100908:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546a69 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.100915:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760752.100918:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546a69 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.100926:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546a69 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.101045:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546a69 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.101059:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546a69 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760752.101257:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495842304:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33323us (33524us total) trans 4294973264 rc 0/0 00000100:00100000:0.0:1789760752.101267:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5976 00000100:00100000:0.0:1789760752.101277:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736357485473 time 736388635519 state 0x2 ref 0 00000100:00100000:1.0:1789760752.113664:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760752.113835:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495842432 00000100:00100000:3.0:1789760752.113848:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495842432 00000100:00100000:3.0:1789760752.113910:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736357485473 time 736388635519 state 0x0 ref 1 00000100:00100000:3.0:1789760752.113927:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736388895935 time 736388635519 state 0x1 ref 1 00000100:00100000:3.0:1789760752.114127:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736388895935 time 736434841531 state 0x0 ref 1 00000100:00100000:3.0:1789760752.114142:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495842432:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760752.114195:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24363008, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:3.0:1789760752.114204:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a70 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.114226:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760752.114230:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.114237:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.114250:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760752.145794:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973265 is committed 00002000:00010000:1.0:1789760752.146101:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24363008 -> 24367104 00010000:00010000:1.0:1789760752.146120:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.146139:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a70 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.146148:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a70 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.146156:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760752.146161:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a70 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.146171:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a70 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.146191:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a70 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.146200:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546a70 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.146396:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495842432:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32252us (32738us total) trans 4294973265 rc 0/0 00000100:00100000:1.0:1789760752.146408:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5977 00000100:00100000:1.0:1789760752.146419:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736388895935 time 736434841531 state 0x2 ref 0 00000100:00100000:1.0:1789760752.151856:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760752.152137:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495842560 00000100:00100000:2.0:1789760752.152146:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495842560 00000100:00100000:2.0:1789760752.152191:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736388895935 time 736434841531 state 0x0 ref 1 00000100:00100000:2.0:1789760752.152206:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736435101947 time 736434841531 state 0x1 ref 1 00000100:00100000:2.0:1789760752.152237:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736435101947 time 736472954356 state 0x0 ref 1 00000100:00100000:2.0:1789760752.152247:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04932680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495842560:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760752.152459:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24367104, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:2.0:1789760752.152469:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546a77 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.152483:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760752.152487:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760752.152493:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546a77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.152506:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546a77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760752.182459:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24367104 -> 24371200 00010000:00010000:2.0:1789760752.182482:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546a77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.182495:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546a77 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.182503:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546a77 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.182510:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760752.182515:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546a77 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.182523:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546a77 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.182549:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546a77 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.182557:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546a77 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760752.182708:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973266 is committed 00000100:00100000:2.0:1789760752.182760:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04932680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495842560:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30508us (30906us total) trans 4294973266 rc 0/0 00000100:00100000:2.0:1789760752.182771:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5978 00000100:00100000:2.0:1789760752.182781:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736435101947 time 736472954356 state 0x2 ref 0 00000100:00100000:2.0:1789760752.193056:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760752.193276:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495842688 00000100:00100000:0.0:1789760752.193312:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495842688 00000100:00100000:0.0:1789760752.193370:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736435101947 time 736472954356 state 0x0 ref 1 00000100:00100000:0.0:1789760752.193387:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736473214772 time 736472954356 state 0x1 ref 1 00000100:00100000:0.0:1789760752.193416:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736473214772 time 736514132706 state 0x0 ref 1 00000100:00100000:0.0:1789760752.193428:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495842688:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760752.193471:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24371200, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:0.0:1789760752.193480:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546a7e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.193497:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760752.193501:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760752.193508:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546a7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.193522:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546a7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760752.244108:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24371200 -> 24375296 00010000:00010000:0.0:1789760752.244127:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546a7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.244140:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546a7e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.244149:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546a7e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.244157:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760752.244162:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546a7e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.244172:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546a7e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.244192:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546a7e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.244200:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546a7e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760752.244333:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495842688:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50902us (51280us total) trans 4294973267 rc 0/0 00000100:00100000:0.0:1789760752.244344:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5979 00000100:00100000:0.0:1789760752.244355:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736473214772 time 736514132706 state 0x2 ref 0 00000001:00080000:1.0:1789760752.244446:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973267 is committed 00000100:00100000:1.0:1789760752.251751:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760752.251892:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495842816 00000100:00100000:3.0:1789760752.251901:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495842816 00000100:00100000:3.0:1789760752.251972:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736473214772 time 736514132706 state 0x0 ref 1 00000100:00100000:3.0:1789760752.251989:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736514393122 time 736514132706 state 0x1 ref 1 00000100:00100000:3.0:1789760752.252020:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736514393122 time 736572736677 state 0x0 ref 1 00000100:00100000:3.0:1789760752.252032:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04930a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495842816:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760752.252083:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24375296, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:3.0:1789760752.252092:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a85 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.252108:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760752.252114:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.252120:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.252138:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760752.298163:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973268 is committed 00002000:00010000:3.0:1789760752.298245:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24375296 -> 24379392 00010000:00010000:3.0:1789760752.298260:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.298273:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a85 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.298372:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a85 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.298380:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760752.298385:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a85 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.298395:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a85 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.298414:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a85 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.298422:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a85 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.298554:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04930a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495842816:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46518us (46806us total) trans 4294973268 rc 0/0 00000100:00100000:3.0:1789760752.298565:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5980 00000100:00100000:3.0:1789760752.298575:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736514393122 time 736572736677 state 0x2 ref 0 00000100:00100000:2.0:1789760752.304845:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760752.304917:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495842944 00000100:00100000:3.0:1789760752.304927:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495842944 00000100:00100000:3.0:1789760752.305052:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736514393122 time 736572736677 state 0x0 ref 1 00000100:00100000:3.0:1789760752.305068:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736572997093 time 736572736677 state 0x1 ref 1 00000100:00100000:3.0:1789760752.305099:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736572997093 time 736625815584 state 0x0 ref 1 00000100:00100000:3.0:1789760752.305111:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495842944:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760752.305156:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24379392, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:3.0:1789760752.305164:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546a8c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.305180:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760752.305185:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.305192:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546a8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.305207:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546a8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760752.333187:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973269 is committed 00002000:00010000:2.0:1789760752.333190:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24379392 -> 24383488 00010000:00010000:2.0:1789760752.333205:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546a8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.333218:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546a8c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.333227:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546a8c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.333236:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760752.333241:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546a8c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.333251:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546a8c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.333271:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546a8c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.333312:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546a8c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760752.333468:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495842944:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28355us (28629us total) trans 4294973269 rc 0/0 00000100:00100000:2.0:1789760752.333479:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5981 00000100:00100000:2.0:1789760752.333488:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736572997093 time 736625815584 state 0x2 ref 0 00000100:00100000:2.0:1789760752.336921:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760752.337038:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495843072 00000100:00100000:0.0:1789760752.337049:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495843072 00000100:00100000:0.0:1789760752.337101:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736572997093 time 736625815584 state 0x0 ref 1 00000100:00100000:0.0:1789760752.337115:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736626076000 time 736625815584 state 0x1 ref 1 00000100:00100000:0.0:1789760752.337145:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736626076000 time 736657861856 state 0x0 ref 1 00000100:00100000:0.0:1789760752.337156:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495843072:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760752.337245:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24383488, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:0.0:1789760752.337253:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546a93 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.337268:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760752.337273:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760752.337315:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546a93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.337442:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546a93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760752.399844:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973270 is committed 00002000:00010000:1.0:1789760752.399978:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24383488 -> 24387584 00010000:00010000:1.0:1789760752.399994:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546a93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.400008:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546a93 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.400018:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546a93 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.400026:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760752.400032:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546a93 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.400042:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546a93 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.400063:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546a93 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.400071:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546a93 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.400164:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495843072:12345-192.168.203.9@tcp:4:dd.0 Request processed in 63005us (63247us total) trans 4294973270 rc 0/0 00000100:00100000:1.0:1789760752.401073:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5982 00000100:00100000:1.0:1789760752.401084:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736626076000 time 736657861856 state 0x2 ref 0 00000100:00100000:1.0:1789760752.408416:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760752.408534:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495843200 00000100:00100000:3.0:1789760752.408544:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495843200 00000100:00100000:3.0:1789760752.408608:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736626076000 time 736657861856 state 0x0 ref 1 00000100:00100000:3.0:1789760752.408622:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736658122272 time 736657861856 state 0x1 ref 1 00000100:00100000:3.0:1789760752.408651:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736658122272 time 736729367922 state 0x0 ref 1 00000100:00100000:3.0:1789760752.408663:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04931180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495843200:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760752.408713:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24387584, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:3.0:1789760752.408722:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a9a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.408738:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760752.408743:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.408750:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.408764:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760752.444043:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24387584 -> 24391680 00010000:00010000:1.0:1789760752.444060:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.444070:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a9a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.444076:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a9a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.444080:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760752.444084:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a9a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.444090:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a9a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.444104:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a9a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.444108:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546a9a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.444267:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495843200:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35602us (35857us total) trans 4294973271 rc 0/0 00000100:00100000:1.0:1789760752.444276:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5983 00000100:00100000:1.0:1789760752.444312:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736658122272 time 736729367922 state 0x2 ref 0 00000001:00080000:0.0:1789760752.444376:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973271 is committed 00000100:00100000:0.0:1789760752.447696:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760752.447789:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495843328 00000100:00100000:2.0:1789760752.447800:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495843328 00000100:00100000:2.0:1789760752.447854:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736658122272 time 736729367922 state 0x0 ref 1 00000100:00100000:2.0:1789760752.447870:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736729628338 time 736729367922 state 0x1 ref 1 00000100:00100000:2.0:1789760752.447931:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736729628338 time 736768647200 state 0x0 ref 1 00000100:00100000:2.0:1789760752.447951:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495843328:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760752.448007:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24391680, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:2.0:1789760752.448017:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546aa1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.448032:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760752.448036:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760752.448042:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546aa1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.448053:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546aa1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.472571:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973272 is committed 00002000:00010000:2.0:1789760752.473052:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24391680 -> 24395776 00010000:00010000:2.0:1789760752.473067:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546aa1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.473080:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546aa1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.473089:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546aa1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.473095:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760752.473100:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546aa1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.473107:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546aa1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.473129:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546aa1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.473136:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546aa1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760752.473226:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495843328:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25272us (25536us total) trans 4294973272 rc 0/0 00000100:00100000:2.0:1789760752.473236:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5984 00000100:00100000:2.0:1789760752.473246:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736729628338 time 736768647200 state 0x2 ref 0 00000100:00100000:2.0:1789760752.477982:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760752.478102:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495843456 00000100:00100000:1.0:1789760752.478112:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495843456 00000100:00100000:1.0:1789760752.478165:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736729628338 time 736768647200 state 0x0 ref 1 00000100:00100000:1.0:1789760752.478180:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736768907616 time 736768647200 state 0x1 ref 1 00000100:00100000:1.0:1789760752.478234:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736768907616 time 736798950736 state 0x0 ref 1 00000100:00100000:1.0:1789760752.478247:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495843456:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760752.478323:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24395776, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:1.0:1789760752.478332:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354546aa8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.478347:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760752.478351:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.478358:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354546aa8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.478371:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354546aa8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.478879:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760752.524196:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24395776 -> 24399872 00010000:00010000:1.0:1789760752.524214:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354546aa8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.524223:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354546aa8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.524229:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354546aa8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.524233:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760752.524237:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354546aa8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.524243:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354546aa8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.524258:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354546aa8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.524262:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354546aa8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.524368:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495843456:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46117us (46390us total) trans 4294973273 rc 0/0 00000100:00100000:1.0:1789760752.524382:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5985 00000100:00100000:1.0:1789760752.524391:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736768907616 time 736798950736 state 0x2 ref 0 00000001:00080000:3.0:1789760752.524468:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973273 is committed 00000100:00100000:1.0:1789760752.528027:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760752.528176:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495843584 00000100:00100000:3.0:1789760752.528186:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495843584 00000100:00100000:3.0:1789760752.528241:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736768907616 time 736798950736 state 0x0 ref 1 00000100:00100000:3.0:1789760752.528257:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736799211152 time 736798950736 state 0x1 ref 1 00000100:00100000:3.0:1789760752.528320:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736799211152 time 736849035837 state 0x0 ref 1 00000100:00100000:3.0:1789760752.528333:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04931f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495843584:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760752.528381:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24399872, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:3.0:1789760752.528391:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546aaf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.528406:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760752.528410:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.528417:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546aaf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.528431:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546aaf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760752.564795:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24399872 -> 24403968 00010000:00010000:3.0:1789760752.564816:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546aaf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.564829:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546aaf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.564839:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546aaf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.564848:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760752.564853:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546aaf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.564862:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546aaf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.564882:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546aaf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.564891:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546aaf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.565046:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495843584:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36709us (37021us total) trans 4294973274 rc 0/0 00000100:00100000:3.0:1789760752.565059:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5986 00000100:00100000:3.0:1789760752.565073:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736799211152 time 736849035837 state 0x2 ref 0 00000001:00080000:3.0:1789760752.565117:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973274 is committed 00000100:00100000:0.0:1789760752.571908:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760752.572347:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495843712 00000100:00100000:2.0:1789760752.572357:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495843712 00000100:00100000:2.0:1789760752.572410:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736799211152 time 736849035837 state 0x0 ref 1 00000100:00100000:2.0:1789760752.572427:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736849296253 time 736849035837 state 0x1 ref 1 00000100:00100000:2.0:1789760752.572455:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736849296253 time 736893171826 state 0x0 ref 1 00000100:00100000:2.0:1789760752.572467:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495843712:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760752.572514:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24403968, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:2.0:1789760752.572522:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546ab6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.572543:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760752.572547:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760752.572553:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546ab6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.572567:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546ab6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760752.613719:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973275 is committed 00002000:00010000:3.0:1789760752.613908:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24403968 -> 24408064 00010000:00010000:3.0:1789760752.613924:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546ab6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.614006:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546ab6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.614024:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546ab6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.614031:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760752.614036:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546ab6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.614047:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546ab6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.614068:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546ab6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.614077:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546ab6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.614208:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495843712:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41738us (42304us total) trans 4294973275 rc 0/0 00000100:00100000:3.0:1789760752.614219:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5987 00000100:00100000:3.0:1789760752.614230:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736849296253 time 736893171826 state 0x2 ref 0 00000100:00100000:2.0:1789760752.618831:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760752.618927:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495843840 00000100:00100000:1.0:1789760752.618936:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495843840 00000100:00100000:1.0:1789760752.618999:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736849296253 time 736893171826 state 0x0 ref 1 00000100:00100000:1.0:1789760752.619014:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736893432242 time 736893171826 state 0x1 ref 1 00000100:00100000:1.0:1789760752.619051:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736893432242 time 736939767780 state 0x0 ref 1 00000100:00100000:1.0:1789760752.619062:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495843840:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760752.619109:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24408064, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:1.0:1789760752.619119:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546abd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.619134:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760752.619139:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.619145:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546abd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.619158:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546abd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760752.646407:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973276 is committed 00002000:00010000:3.0:1789760752.646921:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24408064 -> 24412160 00002000:00010000:3.0:1789760752.646926:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760751 -> 1789760752 00002000:00010000:3.0:1789760752.646929:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760751 -> 1789760752 00010000:00010000:3.0:1789760752.647015:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546abd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.647030:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546abd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.647039:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546abd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.647046:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760752.647050:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546abd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.647059:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546abd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.647090:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546abd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.647099:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546abd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.647202:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495843840:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28136us (28375us total) trans 4294973276 rc 0/0 00000100:00100000:3.0:1789760752.647213:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5988 00000100:00100000:3.0:1789760752.647223:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736893432242 time 736939767780 state 0x2 ref 0 00000100:00100000:1.0:1789760752.651466:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760752.651631:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495843968 00000100:00100000:3.0:1789760752.651641:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495843968 00000100:00100000:3.0:1789760752.651696:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736893432242 time 736939767780 state 0x0 ref 1 00000100:00100000:3.0:1789760752.651711:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736940028196 time 736939767780 state 0x1 ref 1 00000100:00100000:3.0:1789760752.651767:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736940028196 time 736972484122 state 0x0 ref 1 00000100:00100000:3.0:1789760752.651779:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495843968:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760752.651822:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24412160, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:3.0:1789760752.651897:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ac4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.651914:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760752.651919:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.651926:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ac4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.651947:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ac4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760752.706619:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973277 is committed 00002000:00010000:3.0:1789760752.706723:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24412160 -> 24416256 00010000:00010000:3.0:1789760752.706737:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ac4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.706749:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ac4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.706763:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ac4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.706770:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760752.706775:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ac4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.706784:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ac4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.706806:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ac4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.706815:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ac4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.706908:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495843968:12345-192.168.203.9@tcp:4:dd.0 Request processed in 55125us (55446us total) trans 4294973277 rc 0/0 00000100:00100000:3.0:1789760752.706918:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5989 00000100:00100000:3.0:1789760752.706929:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736940028196 time 736972484122 state 0x2 ref 0 00000100:00100000:0.0:1789760752.722143:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760752.722260:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495844096 00000100:00100000:3.0:1789760752.722270:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495844096 00000100:00100000:3.0:1789760752.722360:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736940028196 time 736972484122 state 0x0 ref 1 00000100:00100000:3.0:1789760752.722376:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736972744538 time 736972484122 state 0x1 ref 1 00000100:00100000:3.0:1789760752.722399:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736972744538 time 737043115571 state 0x0 ref 1 00000100:00100000:3.0:1789760752.722412:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495844096:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760752.722462:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24416256, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:3.0:1789760752.722471:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546acb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.722488:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760752.722492:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.722499:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546acb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.722512:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546acb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760752.789810:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973278 is committed 00002000:00010000:3.0:1789760752.789923:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24416256 -> 24420352 00010000:00010000:3.0:1789760752.790017:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546acb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.790030:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546acb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.790040:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546acb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.790047:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760752.790053:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546acb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.790062:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546acb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.790197:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546acb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.790207:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546acb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.790462:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495844096:12345-192.168.203.9@tcp:4:dd.0 Request processed in 68046us (68323us total) trans 4294973278 rc 0/0 00000100:00100000:3.0:1789760752.790473:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5990 00000100:00100000:3.0:1789760752.790482:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736972744538 time 737043115571 state 0x2 ref 0 00000100:00100000:2.0:1789760752.795413:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760752.795570:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495844224 00000100:00100000:3.0:1789760752.795579:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495844224 00000100:00100000:3.0:1789760752.795628:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 736972744538 time 737043115571 state 0x0 ref 1 00000100:00100000:3.0:1789760752.795643:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737043375987 time 737043115571 state 0x1 ref 1 00000100:00100000:3.0:1789760752.795674:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737043375987 time 737116390251 state 0x0 ref 1 00000100:00100000:3.0:1789760752.795686:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495844224:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760752.795732:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24420352, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:3.0:1789760752.795742:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ad2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.795756:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760752.795760:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.795767:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ad2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.795780:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ad2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760752.821117:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24420352 -> 24424448 00010000:00010000:3.0:1789760752.821136:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ad2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.821149:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ad2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.821158:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ad2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.821166:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760752.821170:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ad2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.821178:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ad2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.821201:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ad2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.821208:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ad2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.821378:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495844224:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25689us (25971us total) trans 4294973279 rc 0/0 00000100:00100000:3.0:1789760752.821389:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5991 00000001:00080000:0.0:1789760752.821398:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973279 is committed 00000100:00100000:3.0:1789760752.821398:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737043375987 time 737116390251 state 0x2 ref 0 00000100:00100000:1.0:1789760752.832902:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760752.833053:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495844352 00000100:00100000:3.0:1789760752.833063:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495844352 00000100:00100000:3.0:1789760752.833117:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737043375987 time 737116390251 state 0x0 ref 1 00000100:00100000:3.0:1789760752.833133:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737116650667 time 737116390251 state 0x1 ref 1 00000100:00100000:3.0:1789760752.833168:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737116650667 time 737153885547 state 0x0 ref 1 00000100:00100000:3.0:1789760752.833180:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495844352:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760752.833227:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24424448, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:3.0:1789760752.833236:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ad9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.833251:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760752.833257:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.833264:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ad9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.833320:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ad9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760752.867521:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973280 is committed 00002000:00010000:0.0:1789760752.867586:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24424448 -> 24428544 00010000:00010000:0.0:1789760752.867600:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ad9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.867612:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ad9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.867621:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ad9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.867628:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760752.867633:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ad9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.867643:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ad9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.867661:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ad9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.867670:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546ad9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760752.867763:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495844352:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34580us (34865us total) trans 4294973280 rc 0/0 00000100:00100000:0.0:1789760752.867773:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5992 00000100:00100000:0.0:1789760752.867783:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737116650667 time 737153885547 state 0x2 ref 0 00000100:00100000:1.0:1789760752.872317:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760752.872906:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495844480 00000100:00100000:2.0:1789760752.872915:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495844480 00000100:00100000:2.0:1789760752.872989:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737116650667 time 737153885547 state 0x0 ref 1 00000100:00100000:2.0:1789760752.873003:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737154145963 time 737153885547 state 0x1 ref 1 00000100:00100000:2.0:1789760752.873031:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737154145963 time 737193747792 state 0x0 ref 1 00000100:00100000:2.0:1789760752.873044:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495844480:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760752.873088:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24428544, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:2.0:1789760752.873097:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546ae0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.873112:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760752.873116:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760752.873122:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546ae0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.873135:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546ae0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760752.919633:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24428544 -> 24432640 00010000:00010000:3.0:1789760752.919653:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546ae0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.919666:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546ae0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.919676:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546ae0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.919683:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760752.919688:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546ae0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.919697:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546ae0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.919718:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546ae0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.919727:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546ae0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.919822:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495844480:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46775us (47545us total) trans 4294973281 rc 0/0 00000100:00100000:3.0:1789760752.919832:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5993 00000100:00100000:3.0:1789760752.919843:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737154145963 time 737193747792 state 0x2 ref 0 00000001:00080000:0.0:1789760752.919926:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973281 is committed 00000100:00100000:2.0:1789760752.924129:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760752.924268:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495844608 00000100:00100000:3.0:1789760752.924277:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495844608 00000100:00100000:1.0:1789760752.926740:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff929036f66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3318:1876699520309248:0@lo:400:kworker.0 00000100:00100000:1.0:1789760752.926811:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760752.926867:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f0492ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3318:1876699520309760:0@lo:400:kworker.0 00000100:00100000:1.0:1789760752.926894:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760752.926921:0:3318:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1789760752.926927:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff929036f65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:59c4e264-9b75-482b-8813-3d1dc124f384:3318:1876699520309120:0@lo:400:kworker.0 00000100:00100000:1.0:1789760752.927035:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760752.927060:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff929036f5aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3318:1876699520309632:0@lo:400:kworker.0 00000100:00100000:1.0:1789760752.927083:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760752.927110:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff92903f921c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520309376:0@lo:400:kworker.0 00000100:00100000:1.0:1789760752.927131:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760752.927149:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760752.927154:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff929040814000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520309504:0@lo:400:kworker.0 00000100:00100000:1.0:1789760752.927173:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760752.927200:0:12710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520309248 00000100:00100000:1.0:1789760752.927206:0:12710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520309248 00000100:00100000:1.0:1789760752.927245:0:12710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520309632 00000100:00100000:1.0:1789760752.927248:0:12710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520309632 00000100:00100000:1.0:1789760752.927273:0:12710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520309376 00000100:00100000:1.0:1789760752.928035:0:12710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520309376 00000100:00100000:1.0:1789760752.928073:0:12710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 492 00000100:00100000:1.0:1789760752.928082:0:12710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3318:x1876699520309248:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760752.928153:0:12710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3318:x1876699520309248:12345-0@lo:400:kworker.0 Request processed in 72us (1345us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760752.928164:0:12710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 492 00000100:00100000:1.0:1789760752.928182:0:12710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 493 00000100:00100000:1.0:1789760752.928188:0:12710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01c8d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1876699520309632:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760752.928216:0:12710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01c8d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1876699520309632:12345-0@lo:400:kworker.0 Request processed in 29us (1132us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760752.928222:0:12710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 493 00000100:00100000:1.0:1789760752.928232:0:12710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 494 00000100:00100000:1.0:1789760752.928238:0:12710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520309376:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760752.928271:0:12710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520309376:12345-0@lo:400:kworker.0 Request processed in 33us (1138us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760752.928728:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff929036f5aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3318:1876699520309632:0@lo:400:kworker.0 00000100:00100000:2.0:1789760752.928741:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff929036f66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3318:1876699520309248:0@lo:400:kworker.0 00000100:00100000:1.0:1789760752.928887:0:12710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 494 00000100:00100000:1.0:1789760752.928910:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520309760 00000100:00100000:1.0:1789760752.928915:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520309760 00000100:00100000:1.0:1789760752.929417:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520309504 00000100:00100000:1.0:1789760752.929422:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520309504 00000100:00100000:3.0:1789760752.929570:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737154145963 time 737193747792 state 0x0 ref 1 00000100:00100000:3.0:1789760752.929586:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737194008208 time 737193747792 state 0x1 ref 1 00000100:00100000:3.0:1789760752.929607:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737194008208 time 737250324879 state 0x0 ref 1 00000100:00100000:3.0:1789760752.929617:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495844608:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760752.929657:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24432640, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:3.0:1789760752.929666:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ae7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.929681:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760752.929685:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.929691:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ae7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.929704:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ae7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.930036:0:6709:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 228 00000100:00100000:1.0:1789760752.930044:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01c8c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3318:x1876699520309760:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760752.930107:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01c8c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3318:x1876699520309760:12345-0@lo:400:kworker.0 Request processed in 64us (3211us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760752.930113:0:6709:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 228 00000100:00100000:1.0:1789760752.930126:0:6709:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 229 00000100:00100000:1.0:1789760752.930131:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520309504:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760752.930157:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520309504:12345-0@lo:400:kworker.0 Request processed in 26us (2982us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760752.930162:0:6709:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 229 00000100:00100000:1.0:1789760752.930184:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520309120 00000100:00100000:1.0:1789760752.930189:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520309120 00000100:00100000:1.0:1789760752.930212:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 405 00000100:00100000:1.0:1789760752.930217:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929012310850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:59c4e264-9b75-482b-8813-3d1dc124f384+19:3318:x1876699520309120:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760752.930246:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929012310850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:59c4e264-9b75-482b-8813-3d1dc124f384+19:3318:x1876699520309120:12345-0@lo:400:kworker.0 Request processed in 29us (3211us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760752.930252:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 405 00000100:00080000:0.0:1789760752.930419:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789760752.930424:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789760752.930426:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760752.930619:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff929040814000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520309504:0@lo:400:kworker.0 00000100:00100000:0.0:1789760752.930632:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff92903f921c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520309376:0@lo:400:kworker.0 00000100:00100000:0.0:1789760752.930667:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff929036f65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:59c4e264-9b75-482b-8813-3d1dc124f384:3318:1876699520309120:0@lo:400:kworker.0 00000100:00100000:0.0:1789760752.930678:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f0492ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3318:1876699520309760:0@lo:400:kworker.0 00000001:00080000:1.0:1789760752.965576:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973282 is committed 00002000:00010000:1.0:1789760752.969148:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24432640 -> 24436736 00010000:00010000:1.0:1789760752.969168:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ae7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.969181:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ae7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.969190:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ae7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.969197:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760752.969202:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ae7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.969210:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ae7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.969230:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ae7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.969242:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ae7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.969539:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495844608:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39917us (45414us total) trans 4294973282 rc 0/0 00000100:00100000:1.0:1789760752.969550:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5994 00000100:00100000:1.0:1789760752.969561:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737194008208 time 737250324879 state 0x2 ref 0 00000100:00100000:1.0:1789760752.972270:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760752.972434:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495844736 00000100:00100000:0.0:1789760752.972443:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495844736 00000100:00100000:0.0:1789760752.972498:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737194008208 time 737250324879 state 0x0 ref 1 00000100:00100000:0.0:1789760752.972513:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737250585295 time 737250324879 state 0x1 ref 1 00000100:00100000:0.0:1789760752.972541:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737250585295 time 737293257898 state 0x0 ref 1 00000100:00100000:0.0:1789760752.972554:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495844736:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760752.972601:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24436736, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:0.0:1789760752.972611:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546aee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.972626:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760752.972630:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760752.972637:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546aee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.972649:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546aee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760753.022710:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24436736 -> 24440832 00010000:00010000:1.0:1789760753.022731:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546aee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.022745:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546aee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.022754:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546aee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.022761:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760753.022766:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546aee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.022775:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546aee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.022800:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546aee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.022808:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546aee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.022917:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495844736:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50360us (50651us total) trans 4294973283 rc 0/0 00000100:00100000:1.0:1789760753.022928:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5995 00000100:00100000:1.0:1789760753.022948:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737250585295 time 737293257898 state 0x2 ref 0 00000001:00080000:1.0:1789760753.022998:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973283 is committed 00000100:00100000:2.0:1789760753.027137:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760753.027237:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495844864 00000100:00100000:3.0:1789760753.027247:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495844864 00000100:00100000:3.0:1789760753.027571:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737250585295 time 737293257898 state 0x0 ref 1 00000100:00100000:3.0:1789760753.027587:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737293518314 time 737293257898 state 0x1 ref 1 00000100:00100000:3.0:1789760753.027620:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737293518314 time 737348337460 state 0x0 ref 1 00000100:00100000:3.0:1789760753.027633:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495844864:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760753.027682:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24440832, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:3.0:1789760753.027692:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546af5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.027708:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760753.027712:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.027718:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546af5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.027731:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546af5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760753.032159:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760753.054273:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973284 is committed 00002000:00010000:1.0:1789760753.054380:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24440832 -> 24444928 00010000:00010000:1.0:1789760753.054395:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546af5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.054408:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546af5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.054417:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546af5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.054425:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760753.054430:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546af5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.054621:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546af5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.054642:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546af5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.054651:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546af5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.054813:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495844864:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27176us (27679us total) trans 4294973284 rc 0/0 00000100:00100000:1.0:1789760753.054825:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5996 00000100:00100000:1.0:1789760753.054835:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737293518314 time 737348337460 state 0x2 ref 0 00000100:00100000:2.0:1789760753.060567:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760753.060665:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495844992 00000100:00100000:1.0:1789760753.060747:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495844992 00000100:00100000:1.0:1789760753.060802:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737293518314 time 737348337460 state 0x0 ref 1 00000100:00100000:1.0:1789760753.060817:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737348597876 time 737348337460 state 0x1 ref 1 00000100:00100000:1.0:1789760753.060889:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737348597876 time 737381604889 state 0x0 ref 1 00000100:00100000:1.0:1789760753.060903:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495844992:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760753.060970:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24444928, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:1.0:1789760753.060979:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546afc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.060995:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760753.061000:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.061006:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546afc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.061019:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546afc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760753.098905:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973285 is committed 00002000:00010000:1.0:1789760753.098996:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24444928 -> 24449024 00010000:00010000:1.0:1789760753.099011:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546afc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.099024:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546afc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.099033:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546afc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.099041:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760753.099045:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546afc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.099055:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546afc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.099080:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546afc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.099088:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546afc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.099171:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495844992:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38266us (38610us total) trans 4294973285 rc 0/0 00000100:00100000:1.0:1789760753.099181:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5997 00000100:00100000:1.0:1789760753.099192:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737348597876 time 737381604889 state 0x2 ref 0 00000100:00100000:3.0:1789760753.103204:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760753.103330:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495845120 00000100:00100000:1.0:1789760753.103339:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495845120 00000100:00100000:1.0:1789760753.103404:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737348597876 time 737381604889 state 0x0 ref 1 00000100:00100000:1.0:1789760753.103420:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737381865305 time 737381604889 state 0x1 ref 1 00000100:00100000:1.0:1789760753.103449:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737381865305 time 737424165840 state 0x0 ref 1 00000100:00100000:1.0:1789760753.103461:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495845120:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760753.103513:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24449024, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:1.0:1789760753.103523:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546b03 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.103541:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760753.103546:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.103553:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546b03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.103567:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546b03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760753.138313:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973286 is committed 00002000:00010000:0.0:1789760753.138525:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24449024 -> 24453120 00010000:00010000:0.0:1789760753.138540:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546b03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.138553:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546b03 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.138566:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546b03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.138574:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760753.138579:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546b03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.138588:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546b03 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.138608:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546b03 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.138617:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354546b03 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760753.138746:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495845120:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35282us (35546us total) trans 4294973286 rc 0/0 00000100:00100000:0.0:1789760753.138756:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5998 00000100:00100000:0.0:1789760753.138765:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737381865305 time 737424165840 state 0x2 ref 0 00000100:00100000:1.0:1789760753.144670:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760753.145005:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495845248 00000100:00100000:3.0:1789760753.145016:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495845248 00000100:00100000:3.0:1789760753.145071:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737381865305 time 737424165840 state 0x0 ref 1 00000100:00100000:3.0:1789760753.145086:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737424426256 time 737424165840 state 0x1 ref 1 00000100:00100000:3.0:1789760753.145177:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737424426256 time 737465892840 state 0x0 ref 1 00000100:00100000:3.0:1789760753.145191:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495845248:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760753.145240:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24453120, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:3.0:1789760753.145250:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546b0a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.145265:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760753.145270:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.145515:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546b0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.145532:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546b0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760753.189874:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973287 is committed 00002000:00010000:3.0:1789760753.190003:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24453120 -> 24457216 00010000:00010000:3.0:1789760753.190018:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546b0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.190029:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546b0a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.190037:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546b0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.190060:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760753.190065:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546b0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.190075:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546b0a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.190096:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546b0a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.190105:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546b0a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.190202:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495845248:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45009us (45538us total) trans 4294973287 rc 0/0 00000100:00100000:3.0:1789760753.190213:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 5999 00000100:00100000:3.0:1789760753.190224:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737424426256 time 737465892840 state 0x2 ref 0 00000100:00100000:2.0:1789760753.197878:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760753.198076:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495845376 00000100:00100000:1.0:1789760753.198085:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495845376 00000100:00100000:1.0:1789760753.198136:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737424426256 time 737465892840 state 0x0 ref 1 00000100:00100000:1.0:1789760753.198161:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737466153256 time 737465892840 state 0x1 ref 1 00000100:00100000:1.0:1789760753.198191:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737466153256 time 737518908235 state 0x0 ref 1 00000100:00100000:1.0:1789760753.198203:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495845376:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760753.198251:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24457216, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:1.0:1789760753.198260:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546b11 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.198331:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760753.198337:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.198343:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546b11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.198358:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546b11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760753.243367:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973288 is committed 00002000:00010000:0.0:1789760753.243543:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24457216 -> 24461312 00010000:00010000:0.0:1789760753.243559:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546b11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.243575:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546b11 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.243583:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546b11 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.243590:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760753.243595:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546b11 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.243603:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546b11 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.243623:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546b11 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.243631:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546b11 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760753.243731:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495845376:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45526us (45856us total) trans 4294973288 rc 0/0 00000100:00100000:0.0:1789760753.243741:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6000 00000100:00100000:0.0:1789760753.243750:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737466153256 time 737518908235 state 0x2 ref 0 00000100:00100000:0.0:1789760753.246868:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760753.246986:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495845504 00000100:00100000:1.0:1789760753.246996:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495845504 00000100:00100000:1.0:1789760753.247044:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737466153256 time 737518908235 state 0x0 ref 1 00000100:00100000:1.0:1789760753.247058:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737519168651 time 737518908235 state 0x1 ref 1 00000100:00100000:1.0:1789760753.247088:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737519168651 time 737567805069 state 0x0 ref 1 00000100:00100000:1.0:1789760753.247099:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495845504:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760753.247144:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24461312, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:1.0:1789760753.247153:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b18 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.247167:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760753.247172:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.247178:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.247192:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760753.275042:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973289 is committed 00002000:00010000:1.0:1789760753.275265:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24461312 -> 24465408 00010000:00010000:1.0:1789760753.275404:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.275425:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b18 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.275434:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.275442:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760753.275447:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.275455:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b18 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.275474:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b18 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.275482:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b18 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.275607:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495845504:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28505us (28744us total) trans 4294973289 rc 0/0 00000100:00100000:1.0:1789760753.275618:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6001 00000100:00100000:1.0:1789760753.275627:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737519168651 time 737567805069 state 0x2 ref 0 00000100:00100000:3.0:1789760753.279684:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760753.279788:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495845632 00000100:00100000:1.0:1789760753.279798:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495845632 00000100:00100000:1.0:1789760753.279851:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737519168651 time 737567805069 state 0x0 ref 1 00000100:00100000:1.0:1789760753.279867:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737568065485 time 737567805069 state 0x1 ref 1 00000100:00100000:1.0:1789760753.279948:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737568065485 time 737600664104 state 0x0 ref 1 00000100:00100000:1.0:1789760753.279970:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495845632:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760753.280019:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24465408, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:1.0:1789760753.280028:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546b1f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.280043:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760753.280048:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.280055:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546b1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.280069:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546b1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760753.307061:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24465408 -> 24469504 00010000:00010000:1.0:1789760753.307079:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546b1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.307093:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546b1f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.307107:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546b1f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.307114:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760753.307119:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546b1f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.307129:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546b1f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.307149:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546b1f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.307157:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546b1f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.307243:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495845632:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27271us (27563us total) trans 4294973290 rc 0/0 00000100:00100000:1.0:1789760753.307254:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6002 00000100:00100000:1.0:1789760753.307265:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737568065485 time 737600664104 state 0x2 ref 0 00000001:00080000:3.0:1789760753.307475:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973290 is committed 00000100:00100000:2.0:1789760753.311197:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760753.311500:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495845760 00000100:00100000:0.0:1789760753.311510:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495845760 00000100:00100000:1.0:1789760753.311628:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737568065485 time 737600664104 state 0x0 ref 1 00000100:00100000:1.0:1789760753.311643:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737600924520 time 737600664104 state 0x1 ref 1 00000100:00100000:1.0:1789760753.311710:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737600924520 time 737632426604 state 0x0 ref 1 00000100:00100000:1.0:1789760753.311723:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495845760:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760753.311770:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24469504, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:1.0:1789760753.311779:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b26 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.311795:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760753.311800:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.311807:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.311821:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760753.337494:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24469504 -> 24473600 00010000:00010000:1.0:1789760753.337515:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.337528:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b26 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.337538:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.337546:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760753.337551:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.337561:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b26 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.337580:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b26 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.337589:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b26 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.337672:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495845760:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25947us (26480us total) trans 4294973291 rc 0/0 00000100:00100000:1.0:1789760753.337683:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6003 00000100:00100000:1.0:1789760753.337693:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737600924520 time 737632426604 state 0x2 ref 0 00000001:00080000:1.0:1789760753.337728:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973291 is committed 00000100:00100000:1.0:1789760753.341711:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760753.342307:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495845888 00000100:00100000:2.0:1789760753.342317:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495845888 00000100:00100000:2.0:1789760753.342373:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737600924520 time 737632426604 state 0x0 ref 1 00000100:00100000:2.0:1789760753.342388:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737632687020 time 737632426604 state 0x1 ref 1 00000100:00100000:2.0:1789760753.342472:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737632687020 time 737663188171 state 0x0 ref 1 00000100:00100000:2.0:1789760753.342485:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495845888:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760753.342534:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24473600, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:2.0:1789760753.342543:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546b2d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.342570:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760753.342575:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.342581:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546b2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.342594:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546b2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760753.369555:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973292 is committed 00002000:00010000:2.0:1789760753.369682:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24473600 -> 24477696 00010000:00010000:2.0:1789760753.369698:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546b2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.369711:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546b2d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.369721:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546b2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.369729:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760753.369734:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546b2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.369744:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546b2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.369763:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546b2d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.369772:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546b2d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.369915:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495845888:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27426us (28208us total) trans 4294973292 rc 0/0 00000100:00100000:2.0:1789760753.369926:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6004 00000100:00100000:2.0:1789760753.369946:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737632687020 time 737663188171 state 0x2 ref 0 00000100:00100000:1.0:1789760753.373674:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760753.373770:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495846016 00000100:00100000:3.0:1789760753.373781:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495846016 00000100:00100000:3.0:1789760753.373835:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737632687020 time 737663188171 state 0x0 ref 1 00000100:00100000:3.0:1789760753.373851:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737663448587 time 737663188171 state 0x1 ref 1 00000100:00100000:3.0:1789760753.374211:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737663448587 time 737694925885 state 0x0 ref 1 00000100:00100000:3.0:1789760753.374225:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034ce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495846016:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760753.374338:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24477696, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:3.0:1789760753.374348:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546b34 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.374364:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760753.374369:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.374376:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546b34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.374389:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546b34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760753.401847:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973293 is committed 00002000:00010000:0.0:1789760753.401967:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24477696 -> 24481792 00010000:00010000:0.0:1789760753.401983:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546b34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.401994:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546b34 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.402002:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546b34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.402008:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760753.402012:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546b34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.402020:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546b34 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.402038:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546b34 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.402046:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546b34 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760753.402134:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034ce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495846016:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27907us (28466us total) trans 4294973293 rc 0/0 00000100:00100000:0.0:1789760753.402143:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6005 00000100:00100000:0.0:1789760753.402153:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737663448587 time 737694925885 state 0x2 ref 0 00000100:00100000:0.0:1789760753.405959:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760753.406142:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495846144 00000100:00100000:1.0:1789760753.406152:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495846144 00000100:00100000:1.0:1789760753.406206:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737663448587 time 737694925885 state 0x0 ref 1 00000100:00100000:1.0:1789760753.406221:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737695186301 time 737694925885 state 0x1 ref 1 00000100:00100000:1.0:1789760753.406249:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737695186301 time 737726965978 state 0x0 ref 1 00000100:00100000:1.0:1789760753.406265:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495846144:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760753.406343:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24481792, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:1.0:1789760753.406352:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b3b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.406367:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760753.406372:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.406378:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.406392:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760753.433592:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973294 is committed 00002000:00010000:1.0:1789760753.433803:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24481792 -> 24485888 00010000:00010000:1.0:1789760753.433818:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.433831:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b3b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.433841:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b3b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.433848:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760753.433853:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b3b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.433863:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b3b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.433882:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b3b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.433890:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b3b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.434090:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495846144:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27822us (28141us total) trans 4294973294 rc 0/0 00000100:00100000:1.0:1789760753.434101:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6006 00000100:00100000:1.0:1789760753.434112:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737695186301 time 737726965978 state 0x2 ref 0 00000100:00100000:1.0:1789760753.440604:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760753.440639:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495846272 00000100:00100000:1.0:1789760753.440649:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495846272 00000100:00100000:1.0:1789760753.440699:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737695186301 time 737726965978 state 0x0 ref 1 00000100:00100000:1.0:1789760753.440715:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737727226394 time 737726965978 state 0x1 ref 1 00000100:00100000:1.0:1789760753.440738:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737727226394 time 737761455242 state 0x0 ref 1 00000100:00100000:1.0:1789760753.440750:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495846272:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760753.440796:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24485888, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:1.0:1789760753.440805:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546b42 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.440821:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760753.440825:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.440830:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546b42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.440843:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546b42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760753.441247:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f02769180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520309888:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760753.441312:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760753.442029:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520309888 00000100:00100000:1.0:1789760753.442036:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520309888 00000100:00100000:1.0:1789760753.442072:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 204 00000100:00100000:1.0:1789760753.442081:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520309888:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760753.442108:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760753.442158:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520309888:12345-0@lo:13:osp-pre-1-0.0 Request processed in 78us (845us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760753.442165:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 204 00000100:00100000:0.0:1789760753.443490:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f02769180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520309888:0@lo:13:osp-pre-1-0.0 00000001:00080000:2.0:1789760753.472141:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973295 is committed 00002000:00010000:1.0:1789760753.472238:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24485888 -> 24489984 00010000:00010000:1.0:1789760753.472255:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546b42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.472267:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546b42 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.472375:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546b42 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.472384:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760753.472389:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546b42 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.472398:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546b42 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.472419:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546b42 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.472427:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546b42 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.472512:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495846272:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31759us (31910us total) trans 4294973295 rc 0/0 00000100:00100000:1.0:1789760753.472523:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6007 00000100:00100000:1.0:1789760753.472533:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737727226394 time 737761455242 state 0x2 ref 0 00000100:00100000:1.0:1789760753.477194:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760753.477393:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495846400 00000100:00100000:0.0:1789760753.477402:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495846400 00000100:00100000:0.0:1789760753.477458:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737727226394 time 737761455242 state 0x0 ref 1 00000100:00100000:0.0:1789760753.477473:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737761715658 time 737761455242 state 0x1 ref 1 00000100:00100000:0.0:1789760753.477502:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737761715658 time 737798219058 state 0x0 ref 1 00000100:00100000:0.0:1789760753.477514:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495846400:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760753.477558:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24489984, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:0.0:1789760753.477568:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546b49 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.477585:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760753.477590:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760753.477597:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546b49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.477610:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546b49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.502730:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760753.506258:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973296 is committed 00002000:00010000:3.0:1789760753.506435:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24489984 -> 24494080 00010000:00010000:3.0:1789760753.506452:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546b49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.506465:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546b49 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.506480:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546b49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.506487:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760753.506493:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546b49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.506503:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546b49 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.506521:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546b49 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.506529:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546b49 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.506650:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495846400:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29133us (29460us total) trans 4294973296 rc 0/0 00000100:00100000:3.0:1789760753.506659:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6008 00000100:00100000:3.0:1789760753.506669:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737761715658 time 737798219058 state 0x2 ref 0 00000100:00100000:0.0:1789760753.509825:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760753.510002:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495846528 00000100:00100000:1.0:1789760753.510012:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495846528 00000100:00100000:1.0:1789760753.510065:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737761715658 time 737798219058 state 0x0 ref 1 00000100:00100000:1.0:1789760753.510080:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737798479474 time 737798219058 state 0x1 ref 1 00000100:00100000:1.0:1789760753.510160:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737798479474 time 737830875559 state 0x0 ref 1 00000100:00100000:1.0:1789760753.510174:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495846528:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760753.510241:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24494080, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:1.0:1789760753.510249:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b50 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.510266:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760753.510270:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.510306:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.510320:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760753.534477:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24494080 -> 24498176 00010000:00010000:3.0:1789760753.534498:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.534517:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b50 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.534525:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.534532:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760753.534536:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.534546:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b50 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.534569:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b50 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.534576:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546b50 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.534672:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495846528:12345-192.168.203.9@tcp:4:dd.0 Request processed in 24496us (24850us total) trans 4294973297 rc 0/0 00000100:00100000:3.0:1789760753.534683:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6009 00000100:00100000:3.0:1789760753.534694:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737798479474 time 737830875559 state 0x2 ref 0 00000001:00080000:0.0:1789760753.534766:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973297 is committed 00000100:00100000:1.0:1789760753.540783:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760753.542505:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495846656 00000100:00100000:2.0:1789760753.542518:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495846656 00000100:00100000:2.0:1789760753.542575:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737798479474 time 737830875559 state 0x0 ref 1 00000100:00100000:2.0:1789760753.542599:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737831135975 time 737830875559 state 0x1 ref 1 00000100:00100000:2.0:1789760753.542631:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737831135975 time 737863347752 state 0x0 ref 1 00000100:00100000:2.0:1789760753.542644:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903639a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495846656:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760753.542689:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24498176, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:2.0:1789760753.542698:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546b57 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.542713:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760753.542717:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.542723:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546b57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.542736:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546b57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760753.591691:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973298 is committed 00002000:00010000:1.0:1789760753.591862:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24498176 -> 24502272 00010000:00010000:1.0:1789760753.591877:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546b57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.591889:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546b57 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.591898:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546b57 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.591905:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760753.591909:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546b57 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.591918:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546b57 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.592010:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546b57 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.592089:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546b57 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.592229:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903639a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495846656:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49579us (51447us total) trans 4294973298 rc 0/0 00000100:00100000:1.0:1789760753.592240:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6010 00000100:00100000:1.0:1789760753.592250:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737831135975 time 737863347752 state 0x2 ref 0 00000100:00100000:1.0:1789760753.601771:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760753.601891:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495846784 00000100:00100000:3.0:1789760753.601902:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495846784 00000100:00100000:3.0:1789760753.601981:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737831135975 time 737863347752 state 0x0 ref 1 00000100:00100000:3.0:1789760753.601996:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737863608168 time 737863347752 state 0x1 ref 1 00000100:00100000:3.0:1789760753.602031:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737863608168 time 737922746838 state 0x0 ref 1 00000100:00100000:3.0:1789760753.602043:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495846784:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760753.602092:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24502272, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:3.0:1789760753.602101:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546b5e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.602117:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760753.602121:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.602128:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546b5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.602141:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546b5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760753.640275:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24502272 -> 24506368 00010000:00010000:0.0:1789760753.640313:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546b5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.640327:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546b5e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.640336:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546b5e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.640343:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760753.640347:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546b5e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.640357:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546b5e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.640377:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546b5e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.640384:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546b5e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760753.640929:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973299 is committed 00000100:00100000:0.0:1789760753.641705:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495846784:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39655us (39934us total) trans 4294973299 rc 0/0 00000100:00100000:0.0:1789760753.641722:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6011 00000100:00100000:0.0:1789760753.641732:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737863608168 time 737922746838 state 0x2 ref 0 00000100:00100000:0.0:1789760753.645665:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760753.648573:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495846912 00000100:00100000:1.0:1789760753.648593:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495846912 00000100:00100000:1.0:1789760753.648666:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737863608168 time 737922746838 state 0x0 ref 1 00000100:00100000:1.0:1789760753.648683:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737923007254 time 737922746838 state 0x1 ref 1 00000100:00100000:1.0:1789760753.648718:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737923007254 time 737969434859 state 0x0 ref 1 00000100:00100000:1.0:1789760753.648731:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495846912:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760753.648787:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24506368, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:1.0:1789760753.648797:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546b65 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.648813:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760753.648818:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.648825:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546b65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.648840:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546b65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760753.702531:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973300 is committed 00002000:00010000:2.0:1789760753.702667:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24506368 -> 24510464 00002000:00010000:2.0:1789760753.702671:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760752 -> 1789760753 00002000:00010000:2.0:1789760753.702674:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760752 -> 1789760753 00010000:00010000:2.0:1789760753.702687:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546b65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.702699:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546b65 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.702709:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546b65 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.702716:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760753.702720:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546b65 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.702729:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546b65 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.702751:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546b65 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.702764:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546b65 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.702856:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495846912:12345-192.168.203.9@tcp:4:dd.0 Request processed in 54122us (57196us total) trans 4294973300 rc 0/0 00000100:00100000:2.0:1789760753.702866:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6012 00000100:00100000:2.0:1789760753.702877:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737923007254 time 737969434859 state 0x2 ref 0 00000100:00100000:2.0:1789760753.706129:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760753.706750:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495847040 00000100:00100000:1.0:1789760753.706761:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495847040 00000100:00100000:1.0:1789760753.706818:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737923007254 time 737969434859 state 0x0 ref 1 00000100:00100000:1.0:1789760753.706834:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737969695275 time 737969434859 state 0x1 ref 1 00000100:00100000:1.0:1789760753.706866:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737969695275 time 738027582891 state 0x0 ref 1 00000100:00100000:1.0:1789760753.706879:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495847040:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760753.706939:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24510464, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:1.0:1789760753.706947:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546b6c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.706979:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760753.706983:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.706990:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546b6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.707002:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546b6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760753.802195:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24510464 -> 24514560 00010000:00010000:1.0:1789760753.802217:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546b6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.802231:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546b6c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.802239:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546b6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.802246:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760753.802250:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546b6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.802259:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546b6c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.802297:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546b6c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.802305:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546b6c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.802400:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495847040:12345-192.168.203.9@tcp:4:dd.0 Request processed in 95519us (96276us total) trans 4294973301 rc 0/0 00000100:00100000:1.0:1789760753.802410:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6013 00000100:00100000:1.0:1789760753.802420:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737969695275 time 738027582891 state 0x2 ref 0 00000001:00080000:1.0:1789760753.802459:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973301 is committed 00000100:00100000:1.0:1789760753.814840:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760753.814906:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495847168 00000100:00100000:0.0:1789760753.814916:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495847168 00000100:00100000:0.0:1789760753.815034:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 737969695275 time 738027582891 state 0x0 ref 1 00000100:00100000:0.0:1789760753.815049:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738027843307 time 738027582891 state 0x1 ref 1 00000100:00100000:0.0:1789760753.815117:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738027843307 time 738135832656 state 0x0 ref 1 00000100:00100000:0.0:1789760753.815130:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034ce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495847168:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760753.815525:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24514560, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:0.0:1789760753.815535:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546b73 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.815551:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760753.815556:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760753.815563:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546b73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.815577:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546b73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760753.856734:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973302 is committed 00002000:00010000:3.0:1789760753.856872:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24514560 -> 24518656 00010000:00010000:3.0:1789760753.856888:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546b73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.856901:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546b73 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.856915:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546b73 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.856923:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760753.856940:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546b73 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.856961:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546b73 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.856980:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546b73 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.856988:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546b73 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.857083:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034ce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495847168:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41950us (42248us total) trans 4294973302 rc 0/0 00000100:00100000:3.0:1789760753.857094:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6014 00000100:00100000:3.0:1789760753.857105:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738027843307 time 738135832656 state 0x2 ref 0 00000100:00100000:3.0:1789760753.860664:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760753.860764:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495847296 00000100:00100000:2.0:1789760753.860774:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495847296 00000100:00100000:2.0:1789760753.860829:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738027843307 time 738135832656 state 0x0 ref 1 00000100:00100000:2.0:1789760753.860845:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738136093072 time 738135832656 state 0x1 ref 1 00000100:00100000:2.0:1789760753.860937:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738136093072 time 738181644833 state 0x0 ref 1 00000100:00100000:2.0:1789760753.860960:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495847296:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760753.861019:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24518656, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:2.0:1789760753.861028:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546b7a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.861045:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760753.861049:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.861057:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546b7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.861071:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546b7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760753.891161:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24518656 -> 24522752 00010000:00010000:0.0:1789760753.891178:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546b7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.891188:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546b7a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.891194:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546b7a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.891198:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760753.891202:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546b7a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.891208:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546b7a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.891223:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546b7a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.891227:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546b7a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760753.892028:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973303 is committed 00000100:00100000:0.0:1789760753.892150:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495847296:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31196us (31489us total) trans 4294973303 rc 0/0 00000100:00100000:0.0:1789760753.892163:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6015 00000100:00100000:0.0:1789760753.892173:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738136093072 time 738181644833 state 0x2 ref 0 00000100:00100000:1.0:1789760753.897640:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760753.898479:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495847424 00000100:00100000:2.0:1789760753.898490:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495847424 00000100:00100000:2.0:1789760753.898547:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738136093072 time 738181644833 state 0x0 ref 1 00000100:00100000:2.0:1789760753.898562:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738181905249 time 738181644833 state 0x1 ref 1 00000100:00100000:2.0:1789760753.898767:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738181905249 time 738219481336 state 0x0 ref 1 00000100:00100000:2.0:1789760753.898781:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495847424:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760753.898833:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24522752, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:2.0:1789760753.898843:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546b81 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.898857:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760753.898861:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.898867:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546b81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.898880:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546b81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760753.933539:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973304 is committed 00002000:00010000:3.0:1789760753.933705:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24522752 -> 24526848 00010000:00010000:3.0:1789760753.933723:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546b81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.933743:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546b81 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.933753:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546b81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.933761:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760753.933765:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546b81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.933774:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546b81 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.933793:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546b81 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.933801:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546b81 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.933891:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495847424:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35108us (36256us total) trans 4294973304 rc 0/0 00000100:00100000:3.0:1789760753.933901:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6016 00000100:00100000:3.0:1789760753.933912:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738181905249 time 738219481336 state 0x2 ref 0 00000100:00100000:1.0:1789760753.937146:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760753.937226:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495847552 00000100:00100000:3.0:1789760753.937236:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495847552 00000100:00100000:3.0:1789760753.937323:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738181905249 time 738219481336 state 0x0 ref 1 00000100:00100000:3.0:1789760753.937337:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738219741752 time 738219481336 state 0x1 ref 1 00000100:00100000:3.0:1789760753.937355:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738219741752 time 738258073247 state 0x0 ref 1 00000100:00100000:3.0:1789760753.937367:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495847552:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760753.937410:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24526848, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:3.0:1789760753.937419:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546b88 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.937433:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760753.937437:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.937444:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546b88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.937457:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546b88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760753.994094:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973305 is committed 00002000:00010000:3.0:1789760753.994215:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24526848 -> 24530944 00010000:00010000:3.0:1789760753.994232:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546b88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.994244:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546b88 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.994258:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546b88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.994265:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760753.994270:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546b88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.994296:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546b88 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.994317:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546b88 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.994325:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546b88 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.994415:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495847552:12345-192.168.203.9@tcp:4:dd.0 Request processed in 57046us (57273us total) trans 4294973305 rc 0/0 00000100:00100000:3.0:1789760753.994426:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6017 00000100:00100000:3.0:1789760753.994437:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738219741752 time 738258073247 state 0x2 ref 0 00000100:00100000:2.0:1789760754.002161:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760754.002332:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495847680 00000100:00100000:0.0:1789760754.002343:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495847680 00000100:00100000:0.0:1789760754.002398:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738219741752 time 738258073247 state 0x0 ref 1 00000100:00100000:0.0:1789760754.002413:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738258333663 time 738258073247 state 0x1 ref 1 00000100:00100000:0.0:1789760754.002446:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738258333663 time 738323162070 state 0x0 ref 1 00000100:00100000:0.0:1789760754.002459:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495847680:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760754.002522:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24530944, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:0.0:1789760754.002532:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546b8f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.002548:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760754.002552:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760754.002560:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546b8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.002574:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546b8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760754.057545:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973306 is committed 00002000:00010000:1.0:1789760754.057627:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24530944 -> 24535040 00010000:00010000:1.0:1789760754.057643:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546b8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.057656:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546b8f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.057665:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546b8f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.057672:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760754.057677:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546b8f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.057686:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546b8f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.057705:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546b8f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.057714:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546b8f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.057889:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495847680:12345-192.168.203.9@tcp:4:dd.0 Request processed in 55428us (55734us total) trans 4294973306 rc 0/0 00000100:00100000:1.0:1789760754.057901:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6018 00000100:00100000:1.0:1789760754.057909:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738258333663 time 738323162070 state 0x2 ref 0 00000100:00100000:0.0:1789760754.064174:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760754.064262:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495847808 00000100:00100000:3.0:1789760754.064417:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495847808 00000100:00100000:2.0:1789760754.065053:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738258333663 time 738323162070 state 0x0 ref 1 00000100:00100000:2.0:1789760754.065071:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738323422486 time 738323162070 state 0x1 ref 1 00000100:00100000:2.0:1789760754.065148:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738323422486 time 738385864706 state 0x0 ref 1 00000100:00100000:2.0:1789760754.065162:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495847808:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760754.065212:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24535040, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:2.0:1789760754.065221:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546b96 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.065238:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760754.065242:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.065249:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546b96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.065262:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546b96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760754.116813:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973307 is committed 00002000:00010000:0.0:1789760754.117023:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24535040 -> 24539136 00010000:00010000:0.0:1789760754.117039:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546b96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.117052:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546b96 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.117061:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546b96 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.117068:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760754.117072:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546b96 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.117080:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546b96 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.117098:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546b96 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.117106:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546b96 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760754.117817:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495847808:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52650us (53645us total) trans 4294973307 rc 0/0 00000100:00100000:0.0:1789760754.117836:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6019 00000100:00100000:0.0:1789760754.117845:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738323422486 time 738385864706 state 0x2 ref 0 00000100:00100000:3.0:1789760754.122943:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760754.123042:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495847936 00000100:00100000:0.0:1789760754.123052:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495847936 00000100:00100000:0.0:1789760754.123108:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738323422486 time 738385864706 state 0x0 ref 1 00000100:00100000:0.0:1789760754.123122:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738386125122 time 738385864706 state 0x1 ref 1 00000100:00100000:0.0:1789760754.123148:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738386125122 time 738443865576 state 0x0 ref 1 00000100:00100000:0.0:1789760754.123161:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495847936:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760754.123211:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24539136, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:0.0:1789760754.123219:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546b9d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.123234:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760754.123239:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760754.123246:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546b9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.123260:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546b9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760754.156957:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24539136 -> 24543232 00010000:00010000:3.0:1789760754.156977:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546b9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.156990:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546b9d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.156999:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546b9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.157007:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760754.157011:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546b9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.157020:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546b9d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.157041:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546b9d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.157048:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354546b9d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.157146:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495847936:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33982us (34325us total) trans 4294973308 rc 0/0 00000100:00100000:3.0:1789760754.157157:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6020 00000100:00100000:3.0:1789760754.157168:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738386125122 time 738443865576 state 0x2 ref 0 00000001:00080000:1.0:1789760754.157237:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973308 is committed 00000100:00100000:2.0:1789760754.161783:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760754.161883:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495848064 00000100:00100000:1.0:1789760754.161893:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495848064 00000100:00100000:1.0:1789760754.161967:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738386125122 time 738443865576 state 0x0 ref 1 00000100:00100000:1.0:1789760754.161984:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738444125992 time 738443865576 state 0x1 ref 1 00000100:00100000:1.0:1789760754.162014:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738444125992 time 738482730708 state 0x0 ref 1 00000100:00100000:1.0:1789760754.162026:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495848064:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760754.162087:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24543232, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:1.0:1789760754.162097:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546ba4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.162113:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760754.162117:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760754.162124:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546ba4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.162138:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546ba4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760754.167200:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760754.167250:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760754.167275:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760754.167326:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495848192 00000100:00100000:0.0:1789760754.167335:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495848192 00000100:00100000:0.0:1789760754.167380:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 406 00000100:00100000:0.0:1789760754.167388:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929012310c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2358:x1876699495848192:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:0.0:1789760754.167454:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929012310c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2358:x1876699495848192:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 66us (203us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760754.167462:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 406 00000100:00100000:0.0:1789760754.167494:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495848320 00000100:00100000:0.0:1789760754.167498:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495848320 00000100:00100000:0.0:1789760754.167529:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495848448 00000100:00100000:0.0:1789760754.167532:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495848448 00000100:00100000:0.0:1789760754.167549:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 495 00000100:00100000:0.0:1789760754.167555:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:17022965-1e4f-46fb-915c-61730096b8e2+8:2358:x1876699495848320:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:0.0:1789760754.167582:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:17022965-1e4f-46fb-915c-61730096b8e2+8:2358:x1876699495848320:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 27us (394us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760754.167587:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 495 00000100:00100000:0.0:1789760754.167598:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 496 00000100:00100000:0.0:1789760754.167603:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2358:x1876699495848448:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:0.0:1789760754.167621:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2358:x1876699495848448:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 19us (346us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760754.167627:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 496 00000100:00080000:0.0:1789760754.167753:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:2.0:1789760754.195876:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973309 is committed 00002000:00010000:1.0:1789760754.196341:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24543232 -> 24547328 00010000:00010000:1.0:1789760754.196355:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546ba4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.196368:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546ba4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.196377:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546ba4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.196384:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760754.196389:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546ba4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.196397:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546ba4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.196418:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546ba4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.196426:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546ba4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.196522:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495848064:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34493us (34744us total) trans 4294973309 rc 0/0 00000100:00100000:1.0:1789760754.196533:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6021 00000100:00100000:1.0:1789760754.196543:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738444125992 time 738482730708 state 0x2 ref 0 00000100:00100000:0.0:1789760754.200534:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760754.200636:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495848576 00000100:00100000:2.0:1789760754.200647:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495848576 00000100:00100000:2.0:1789760754.200699:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738444125992 time 738482730708 state 0x0 ref 1 00000100:00100000:2.0:1789760754.200715:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738482991124 time 738482730708 state 0x1 ref 1 00000100:00100000:2.0:1789760754.200743:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738482991124 time 738521459610 state 0x0 ref 1 00000100:00100000:2.0:1789760754.200757:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495848576:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760754.200806:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24547328, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:2.0:1789760754.200815:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546bab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.200831:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760754.200835:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.200842:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546bab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.200857:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546bab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760754.253671:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973310 is committed 00002000:00010000:3.0:1789760754.253786:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24547328 -> 24551424 00010000:00010000:3.0:1789760754.253802:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546bab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.253813:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546bab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.253822:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546bab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.253829:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760754.253834:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546bab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.253843:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546bab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.253862:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546bab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.253870:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546bab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.253983:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495848576:12345-192.168.203.9@tcp:4:dd.0 Request processed in 53223us (53451us total) trans 4294973310 rc 0/0 00000100:00100000:3.0:1789760754.253993:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6022 00000100:00100000:3.0:1789760754.254004:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738482991124 time 738521459610 state 0x2 ref 0 00000100:00100000:2.0:1789760754.258486:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760754.258642:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495848704 00000100:00100000:3.0:1789760754.258652:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495848704 00000100:00100000:3.0:1789760754.258704:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738482991124 time 738521459610 state 0x0 ref 1 00000100:00100000:3.0:1789760754.258719:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738521720026 time 738521459610 state 0x1 ref 1 00000100:00100000:3.0:1789760754.258749:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738521720026 time 738579466290 state 0x0 ref 1 00000100:00100000:3.0:1789760754.258761:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495848704:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760754.258808:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24551424, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:3.0:1789760754.258817:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546bb2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.258832:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760754.258837:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.258843:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546bb2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.258858:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546bb2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760754.294108:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973311 is committed 00002000:00010000:0.0:1789760754.294217:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24551424 -> 24555520 00010000:00010000:0.0:1789760754.294232:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546bb2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.294245:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546bb2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.294255:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546bb2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.294261:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760754.294265:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546bb2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.294273:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546bb2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.294358:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546bb2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.294366:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546bb2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760754.294487:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495848704:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35723us (36006us total) trans 4294973311 rc 0/0 00000100:00100000:0.0:1789760754.294497:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6023 00000100:00100000:0.0:1789760754.294507:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738521720026 time 738579466290 state 0x2 ref 0 00000100:00100000:0.0:1789760754.299032:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760754.300413:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495848832 00000100:00100000:1.0:1789760754.300426:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495848832 00000100:00100000:1.0:1789760754.300498:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738521720026 time 738579466290 state 0x0 ref 1 00000100:00100000:1.0:1789760754.300513:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738579726706 time 738579466290 state 0x1 ref 1 00000100:00100000:1.0:1789760754.300550:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738579726706 time 738621266478 state 0x0 ref 1 00000100:00100000:1.0:1789760754.300563:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495848832:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760754.300618:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24555520, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:1.0:1789760754.300627:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bb9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.300647:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760754.300651:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760754.300657:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bb9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.300671:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bb9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760754.342142:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973312 is committed 00002000:00010000:1.0:1789760754.342325:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24555520 -> 24559616 00010000:00010000:1.0:1789760754.342341:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bb9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.342354:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bb9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.342362:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bb9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.342368:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760754.342372:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bb9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.342381:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bb9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.342401:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bb9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.342409:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bb9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.342496:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495848832:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41929us (43468us total) trans 4294973312 rc 0/0 00000100:00100000:1.0:1789760754.342506:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6024 00000100:00100000:1.0:1789760754.342516:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738579726706 time 738621266478 state 0x2 ref 0 00000100:00100000:3.0:1789760754.353922:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760754.354089:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495848960 00000100:00100000:2.0:1789760754.354099:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495848960 00000100:00100000:2.0:1789760754.354151:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738579726706 time 738621266478 state 0x0 ref 1 00000100:00100000:2.0:1789760754.354166:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738621526894 time 738621266478 state 0x1 ref 1 00000100:00100000:2.0:1789760754.354202:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738621526894 time 738674917979 state 0x0 ref 1 00000100:00100000:2.0:1789760754.354214:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495848960:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760754.354260:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24559616, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:2.0:1789760754.354314:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bc0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.354331:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760754.354336:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.354342:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.354355:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760754.393511:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973313 is committed 00002000:00010000:1.0:1789760754.393971:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24559616 -> 24563712 00010000:00010000:1.0:1789760754.393987:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.394001:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bc0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.394010:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bc0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.394017:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760754.394022:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bc0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.394032:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bc0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.394056:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bc0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.394064:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bc0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.394164:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495848960:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39946us (40245us total) trans 4294973313 rc 0/0 00000100:00100000:1.0:1789760754.395103:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6025 00000100:00100000:1.0:1789760754.395114:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738621526894 time 738674917979 state 0x2 ref 0 00000100:00100000:2.0:1789760754.405618:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760754.405783:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495849088 00000100:00100000:0.0:1789760754.405793:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495849088 00000100:00100000:0.0:1789760754.405846:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738621526894 time 738674917979 state 0x0 ref 1 00000100:00100000:0.0:1789760754.405864:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738675178395 time 738674917979 state 0x1 ref 1 00000100:00100000:0.0:1789760754.405899:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738675178395 time 738726615390 state 0x0 ref 1 00000100:00100000:0.0:1789760754.405911:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495849088:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760754.405977:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24563712, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:0.0:1789760754.405986:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546bc7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.406002:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760754.406006:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760754.406012:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546bc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.406026:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546bc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760754.445038:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973314 is committed 00002000:00010000:1.0:1789760754.445252:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24563712 -> 24567808 00010000:00010000:1.0:1789760754.445268:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546bc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.445310:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546bc7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.445320:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546bc7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.445328:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760754.445333:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546bc7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.445349:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546bc7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.445370:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546bc7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.445378:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546bc7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.445461:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495849088:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39548us (39849us total) trans 4294973314 rc 0/0 00000100:00100000:1.0:1789760754.445471:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6026 00000100:00100000:1.0:1789760754.445481:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738675178395 time 738726615390 state 0x2 ref 0 00000100:00100000:0.0:1789760754.457521:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760754.459119:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495849216 00000100:00100000:1.0:1789760754.459132:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495849216 00000100:00100000:1.0:1789760754.459193:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738675178395 time 738726615390 state 0x0 ref 1 00000100:00100000:1.0:1789760754.459210:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738726875806 time 738726615390 state 0x1 ref 1 00000100:00100000:1.0:1789760754.459403:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738726875806 time 738780118421 state 0x0 ref 1 00000100:00100000:1.0:1789760754.459418:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f921c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495849216:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760754.459467:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24567808, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:1.0:1789760754.459476:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.459493:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760754.459498:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760754.459504:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.459519:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.526686:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760754.536548:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973315 is committed 00002000:00010000:1.0:1789760754.536723:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24567808 -> 24571904 00010000:00010000:1.0:1789760754.536739:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.536753:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.536762:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.536769:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760754.536774:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.536785:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.536812:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.536821:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.536915:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f921c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495849216:12345-192.168.203.9@tcp:4:dd.0 Request processed in 77496us (79399us total) trans 4294973315 rc 0/0 00000100:00100000:1.0:1789760754.536926:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6027 00000100:00100000:1.0:1789760754.536935:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738726875806 time 738780118421 state 0x2 ref 0 00000100:00100000:2.0:1789760754.545345:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760754.545679:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495849344 00000100:00100000:1.0:1789760754.545689:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495849344 00000100:00100000:1.0:1789760754.545741:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738726875806 time 738780118421 state 0x0 ref 1 00000100:00100000:1.0:1789760754.545754:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738780378837 time 738780118421 state 0x1 ref 1 00000100:00100000:1.0:1789760754.545784:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738780378837 time 738866501281 state 0x0 ref 1 00000100:00100000:1.0:1789760754.545795:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92901237ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495849344:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760754.545842:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24571904, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:1.0:1789760754.545850:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bd5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.545866:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760754.545871:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760754.545878:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bd5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.545891:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bd5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760754.612140:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973316 is committed 00002000:00010000:2.0:1789760754.612547:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24571904 -> 24576000 00010000:00010000:2.0:1789760754.612563:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bd5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.612575:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bd5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.612759:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bd5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.612767:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760754.612772:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bd5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.612781:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bd5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.612800:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bd5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.612807:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546bd5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.613064:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92901237ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495849344:12345-192.168.203.9@tcp:4:dd.0 Request processed in 67088us (67547us total) trans 4294973316 rc 0/0 00000100:00100000:2.0:1789760754.613075:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6028 00000100:00100000:2.0:1789760754.613085:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738780378837 time 738866501281 state 0x2 ref 0 00000100:00100000:2.0:1789760754.622601:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760754.622758:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495849472 00000100:00100000:1.0:1789760754.622767:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495849472 00000100:00100000:1.0:1789760754.622822:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738780378837 time 738866501281 state 0x0 ref 1 00000100:00100000:1.0:1789760754.622838:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738866761697 time 738866501281 state 0x1 ref 1 00000100:00100000:1.0:1789760754.622869:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738866761697 time 738943585853 state 0x0 ref 1 00000100:00100000:1.0:1789760754.622881:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495849472:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760754.622965:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24576000, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:1.0:1789760754.622975:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546bdc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.622990:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760754.622994:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760754.623001:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546bdc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.623014:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546bdc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.653831:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973317 is committed 00002000:00010000:2.0:1789760754.653930:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24576000 -> 24580096 00002000:00010000:2.0:1789760754.653935:0:13059:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760753 -> 1789760754 00002000:00010000:2.0:1789760754.653938:0:13059:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760753 -> 1789760754 00010000:00010000:2.0:1789760754.653962:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546bdc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.653976:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546bdc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.653986:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546bdc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.653994:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760754.653999:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546bdc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.654009:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546bdc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.654027:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546bdc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.654035:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546bdc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.654123:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495849472:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31240us (31538us total) trans 4294973317 rc 0/0 00000100:00100000:2.0:1789760754.654132:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6029 00000100:00100000:2.0:1789760754.654142:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738866761697 time 738943585853 state 0x2 ref 0 00000100:00100000:0.0:1789760754.659705:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760754.659804:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495849600 00000100:00100000:1.0:1789760754.659813:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495849600 00000100:00100000:1.0:1789760754.659869:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738866761697 time 738943585853 state 0x0 ref 1 00000100:00100000:1.0:1789760754.659885:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738943846269 time 738943585853 state 0x1 ref 1 00000100:00100000:1.0:1789760754.659914:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738943846269 time 738980631208 state 0x0 ref 1 00000100:00100000:1.0:1789760754.659935:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495849600:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760754.659989:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24580096, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:1.0:1789760754.659998:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546be3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.660015:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760754.660020:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760754.660027:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546be3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.660041:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546be3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760754.693236:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973318 is committed 00002000:00010000:2.0:1789760754.693539:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24580096 -> 24584192 00010000:00010000:2.0:1789760754.693554:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546be3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.693568:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546be3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.693579:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546be3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.693587:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760754.693592:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546be3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.693602:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546be3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.693624:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546be3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.693632:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546be3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.693761:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495849600:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33832us (34061us total) trans 4294973318 rc 0/0 00000100:00100000:2.0:1789760754.693772:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6030 00000100:00100000:2.0:1789760754.693783:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738943846269 time 738980631208 state 0x2 ref 0 00000100:00100000:0.0:1789760754.697873:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760754.698151:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495849728 00000100:00100000:2.0:1789760754.698161:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495849728 00000100:00100000:2.0:1789760754.698391:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738943846269 time 738980631208 state 0x0 ref 1 00000100:00100000:2.0:1789760754.698407:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738980891624 time 738980631208 state 0x1 ref 1 00000100:00100000:1.0:1789760754.698524:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738980891624 time 739019237211 state 0x0 ref 1 00000100:00100000:1.0:1789760754.698543:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495849728:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760754.698596:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24584192, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:1.0:1789760754.698606:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546bea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.698622:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760754.698628:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760754.698634:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546bea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.698647:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546bea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.726777:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973319 is committed 00002000:00010000:2.0:1789760754.726983:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24584192 -> 24588288 00010000:00010000:2.0:1789760754.726999:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546bea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.727012:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546bea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.727021:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546bea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.727029:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760754.727034:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546bea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.727043:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546bea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.727064:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546bea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.727078:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546bea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.727172:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495849728:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28630us (29304us total) trans 4294973319 rc 0/0 00000100:00100000:2.0:1789760754.727183:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6031 00000100:00100000:2.0:1789760754.727192:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738980891624 time 739019237211 state 0x2 ref 0 00000100:00100000:3.0:1789760754.730492:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760754.730579:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495849856 00000100:00100000:2.0:1789760754.730590:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495849856 00000100:00100000:2.0:1789760754.730646:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 738980891624 time 739019237211 state 0x0 ref 1 00000100:00100000:2.0:1789760754.730660:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739019497627 time 739019237211 state 0x1 ref 1 00000100:00100000:2.0:1789760754.730699:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739019497627 time 739051407873 state 0x0 ref 1 00000100:00100000:2.0:1789760754.730712:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495849856:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760754.730765:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24588288, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:2.0:1789760754.730774:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bf1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.730792:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760754.730796:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.730803:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bf1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.730822:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bf1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.759864:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973320 is committed 00002000:00010000:2.0:1789760754.759972:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24588288 -> 24592384 00010000:00010000:2.0:1789760754.759987:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bf1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.759999:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bf1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.760009:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bf1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.760016:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760754.760020:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bf1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.760028:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bf1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.760047:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bf1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.760054:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bf1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.760184:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495849856:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29469us (29696us total) trans 4294973320 rc 0/0 00000100:00100000:2.0:1789760754.760195:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6032 00000100:00100000:2.0:1789760754.760206:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739019497627 time 739051407873 state 0x2 ref 0 00000100:00100000:0.0:1789760754.765433:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760754.767421:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495849984 00000100:00100000:2.0:1789760754.767434:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495849984 00000100:00100000:3.0:1789760754.768183:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739019497627 time 739051407873 state 0x0 ref 1 00000100:00100000:3.0:1789760754.768202:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739051668289 time 739051407873 state 0x1 ref 1 00000100:00100000:3.0:1789760754.768244:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739051668289 time 739088960461 state 0x0 ref 1 00000100:00100000:3.0:1789760754.768259:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495849984:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760754.768676:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24592384, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:3.0:1789760754.768685:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546bf8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.768701:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760754.768707:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.768713:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546bf8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.768726:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546bf8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760754.802006:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973321 is committed 00002000:00010000:0.0:1789760754.802189:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24592384 -> 24596480 00010000:00010000:0.0:1789760754.802206:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546bf8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.802219:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546bf8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.802229:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546bf8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.802236:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760754.802241:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546bf8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.802250:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546bf8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.802269:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546bf8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.802306:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546bf8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760754.802396:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495849984:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34137us (36968us total) trans 4294973321 rc 0/0 00000100:00100000:0.0:1789760754.802406:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6033 00000100:00100000:0.0:1789760754.802416:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739051668289 time 739088960461 state 0x2 ref 0 00000100:00100000:0.0:1789760754.807869:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760754.807978:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495850112 00000100:00100000:2.0:1789760754.807989:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495850112 00000100:00100000:2.0:1789760754.808042:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739051668289 time 739088960461 state 0x0 ref 1 00000100:00100000:2.0:1789760754.808057:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739089220877 time 739088960461 state 0x1 ref 1 00000100:00100000:2.0:1789760754.808115:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739089220877 time 739128831548 state 0x0 ref 1 00000100:00100000:2.0:1789760754.808129:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495850112:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760754.808180:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24596480, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:2.0:1789760754.808189:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.808205:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760754.808210:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.808216:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.808235:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760754.838965:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973322 is committed 00002000:00010000:0.0:1789760754.839235:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24596480 -> 24600576 00010000:00010000:0.0:1789760754.839250:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.839263:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.839271:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.839300:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760754.839304:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.839314:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.839333:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.839341:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546bff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760754.839429:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495850112:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31297us (31564us total) trans 4294973322 rc 0/0 00000100:00100000:0.0:1789760754.839439:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6034 00000100:00100000:0.0:1789760754.839449:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739089220877 time 739128831548 state 0x2 ref 0 00000100:00100000:3.0:1789760754.846362:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760754.846503:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495850240 00000100:00100000:2.0:1789760754.846514:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495850240 00000100:00100000:2.0:1789760754.846567:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739089220877 time 739128831548 state 0x0 ref 1 00000100:00100000:2.0:1789760754.846581:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739129091964 time 739128831548 state 0x1 ref 1 00000100:00100000:2.0:1789760754.846614:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739129091964 time 739167331848 state 0x0 ref 1 00000100:00100000:2.0:1789760754.846625:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495850240:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760754.846685:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24600576, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:2.0:1789760754.846693:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546c06 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.846709:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760754.846714:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.846721:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546c06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.846736:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546c06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760754.868740:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973323 is committed 00002000:00010000:2.0:1789760754.868910:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24600576 -> 24604672 00010000:00010000:2.0:1789760754.868995:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546c06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.869009:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546c06 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.869018:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546c06 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.869025:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760754.869029:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546c06 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.869038:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546c06 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.869062:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546c06 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.869068:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546c06 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.869311:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495850240:12345-192.168.203.9@tcp:4:dd.0 Request processed in 22649us (22925us total) trans 4294973323 rc 0/0 00000100:00100000:2.0:1789760754.869326:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6035 00000100:00100000:2.0:1789760754.869335:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739129091964 time 739167331848 state 0x2 ref 0 00000100:00100000:2.0:1789760754.878895:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760754.879078:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495850368 00000100:00100000:0.0:1789760754.879088:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495850368 00000100:00100000:0.0:1789760754.879143:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739129091964 time 739167331848 state 0x0 ref 1 00000100:00100000:0.0:1789760754.879159:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739167592264 time 739167331848 state 0x1 ref 1 00000100:00100000:0.0:1789760754.879240:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739167592264 time 739199956407 state 0x0 ref 1 00000100:00100000:0.0:1789760754.879252:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495850368:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760754.879329:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24604672, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:0.0:1789760754.879337:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546c0d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.879352:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760754.879356:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760754.879362:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546c0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.879374:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546c0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760754.929583:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24604672 -> 24608768 00010000:00010000:3.0:1789760754.929603:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546c0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.929616:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546c0d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.929632:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546c0d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.929639:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94aa00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760754.929644:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546c0d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.929653:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546c0d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.929673:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546c0d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.929690:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546c0d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.929858:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495850368:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50601us (50965us total) trans 4294973324 rc 0/0 00000100:00100000:3.0:1789760754.929869:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6036 00000100:00100000:3.0:1789760754.929881:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739167592264 time 739199956407 state 0x2 ref 0 00000001:00080000:0.0:1789760754.929971:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973324 is committed 00000100:00100000:1.0:1789760754.938073:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760754.938341:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495850496 00000100:00100000:0.0:1789760754.938353:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495850496 00000100:00100000:0.0:1789760754.938412:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739167592264 time 739199956407 state 0x0 ref 1 00000100:00100000:0.0:1789760754.938426:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739200216823 time 739199956407 state 0x1 ref 1 00000100:00100000:0.0:1789760754.938450:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739200216823 time 739259166879 state 0x0 ref 1 00000100:00100000:0.0:1789760754.938463:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495850496:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760754.938514:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24608768, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:0.0:1789760754.938525:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546c14 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.938542:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760754.938548:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760754.938555:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546c14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.938568:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546c14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760754.992432:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973325 is committed 00002000:00010000:1.0:1789760754.992665:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24608768 -> 24612864 00010000:00010000:1.0:1789760754.992681:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546c14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.992693:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546c14 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.992703:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546c14 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.992710:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ae00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760754.992716:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546c14 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.992730:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546c14 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.992750:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546c14 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.992758:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546c14 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.992894:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495850496:12345-192.168.203.9@tcp:4:dd.0 Request processed in 54428us (54825us total) trans 4294973325 rc 0/0 00000100:00100000:1.0:1789760754.992905:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6037 00000100:00100000:1.0:1789760754.992915:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739200216823 time 739259166879 state 0x2 ref 0 00000100:00100000:2.0:1789760755.001546:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760755.001652:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495850624 00000100:00100000:0.0:1789760755.001773:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495850624 00000100:00100000:0.0:1789760755.002068:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739200216823 time 739259166879 state 0x0 ref 1 00000100:00100000:0.0:1789760755.002083:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739259427295 time 739259166879 state 0x1 ref 1 00000100:00100000:0.0:1789760755.002154:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739259427295 time 739322869323 state 0x0 ref 1 00000100:00100000:0.0:1789760755.002169:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495850624:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760755.002217:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24612864, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:0.0:1789760755.002227:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546c1b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.002242:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760755.002246:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.002252:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546c1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.002300:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546c1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760755.003186:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760755.078441:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973326 is committed 00002000:00010000:3.0:1789760755.078594:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24612864 -> 24616960 00010000:00010000:3.0:1789760755.078611:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546c1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.078630:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546c1b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.078640:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546c1b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.078647:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760755.078652:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546c1b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.078661:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546c1b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.078681:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546c1b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.078688:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546c1b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.078776:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495850624:12345-192.168.203.9@tcp:4:dd.0 Request processed in 76605us (77233us total) trans 4294973326 rc 0/0 00000100:00100000:3.0:1789760755.078788:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6038 00000100:00100000:3.0:1789760755.078798:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739259427295 time 739322869323 state 0x2 ref 0 00000100:00100000:2.0:1789760755.085155:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760755.086063:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495850752 00000100:00100000:3.0:1789760755.086075:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495850752 00000100:00100000:3.0:1789760755.086136:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739259427295 time 739322869323 state 0x0 ref 1 00000100:00100000:3.0:1789760755.086153:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739323129739 time 739322869323 state 0x1 ref 1 00000100:00100000:3.0:1789760755.086184:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739323129739 time 739406901191 state 0x0 ref 1 00000100:00100000:3.0:1789760755.086197:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495850752:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760755.086240:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24616960, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:3.0:1789760755.086249:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546c22 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.086265:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760755.086269:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.086275:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546c22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.086309:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546c22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760755.179701:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973327 is committed 00002000:00010000:0.0:1789760755.179784:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24616960 -> 24621056 00010000:00010000:0.0:1789760755.179802:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546c22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.179816:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546c22 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.179824:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546c22 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.179831:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760755.179837:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546c22 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.179845:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546c22 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.179868:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546c22 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.179876:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546c22 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760755.180500:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495850752:12345-192.168.203.9@tcp:4:dd.0 Request processed in 94299us (95350us total) trans 4294973327 rc 0/0 00000100:00100000:0.0:1789760755.180511:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6039 00000100:00100000:0.0:1789760755.180521:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739323129739 time 739406901191 state 0x2 ref 0 00000100:00100000:0.0:1789760755.183498:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760755.183638:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495850880 00000100:00100000:1.0:1789760755.183648:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495850880 00000100:00100000:1.0:1789760755.183705:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739323129739 time 739406901191 state 0x0 ref 1 00000100:00100000:1.0:1789760755.183720:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739407161607 time 739406901191 state 0x1 ref 1 00000100:00100000:1.0:1789760755.183751:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739407161607 time 739504467762 state 0x0 ref 1 00000100:00100000:1.0:1789760755.183763:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495850880:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760755.183814:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24621056, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:1.0:1789760755.183824:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546c29 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.183839:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760755.183844:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760755.183850:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546c29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.183863:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546c29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760755.232454:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973328 is committed 00002000:00010000:2.0:1789760755.232646:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24621056 -> 24625152 00010000:00010000:2.0:1789760755.232670:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546c29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.232683:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546c29 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.232691:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546c29 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.232807:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760755.232813:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546c29 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.232822:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546c29 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.232842:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546c29 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.232854:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546c29 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.233128:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495850880:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49362us (49632us total) trans 4294973328 rc 0/0 00000100:00100000:2.0:1789760755.233139:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6040 00000100:00100000:2.0:1789760755.233149:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739407161607 time 739504467762 state 0x2 ref 0 00000100:00100000:1.0:1789760755.246687:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760755.248238:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495851008 00000100:00100000:3.0:1789760755.248254:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495851008 00000100:00100000:3.0:1789760755.248443:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739407161607 time 739504467762 state 0x0 ref 1 00000100:00100000:3.0:1789760755.248460:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739504728178 time 739504467762 state 0x1 ref 1 00000100:00100000:3.0:1789760755.248834:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739504728178 time 739569548516 state 0x0 ref 1 00000100:00100000:3.0:1789760755.248853:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495851008:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760755.248904:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24625152, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:3.0:1789760755.248913:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546c30 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.248950:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760755.248955:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.248962:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546c30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.248977:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546c30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760755.289513:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973329 is committed 00002000:00010000:3.0:1789760755.289830:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24625152 -> 24629248 00010000:00010000:3.0:1789760755.289846:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546c30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.289858:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546c30 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.289867:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546c30 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.289873:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760755.289878:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546c30 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.289887:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546c30 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.289906:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546c30 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.289914:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546c30 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.290021:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495851008:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41167us (43338us total) trans 4294973329 rc 0/0 00000100:00100000:3.0:1789760755.290031:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6041 00000100:00100000:3.0:1789760755.290041:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739504728178 time 739569548516 state 0x2 ref 0 00000100:00100000:1.0:1789760755.298228:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760755.298379:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495851136 00000100:00100000:0.0:1789760755.298388:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495851136 00000100:00100000:0.0:1789760755.298571:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739504728178 time 739569548516 state 0x0 ref 1 00000100:00100000:0.0:1789760755.298599:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739569808932 time 739569548516 state 0x1 ref 1 00000100:00100000:0.0:1789760755.298665:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739569808932 time 739619380861 state 0x0 ref 1 00000100:00100000:0.0:1789760755.298678:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495851136:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760755.298722:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24629248, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:0.0:1789760755.298732:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546c37 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.298748:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760755.298753:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.298759:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546c37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.298774:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546c37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760755.348727:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973330 is committed 00002000:00010000:3.0:1789760755.348833:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24629248 -> 24633344 00010000:00010000:3.0:1789760755.348848:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546c37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.348860:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546c37 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.348870:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546c37 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.348876:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760755.348882:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546c37 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.348891:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546c37 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.348909:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546c37 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.348926:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546c37 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.349022:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495851136:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50341us (50797us total) trans 4294973330 rc 0/0 00000100:00100000:3.0:1789760755.349032:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6042 00000100:00100000:3.0:1789760755.349041:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739569808932 time 739619380861 state 0x2 ref 0 00000100:00100000:0.0:1789760755.355212:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760755.355324:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495851264 00000100:00100000:2.0:1789760755.355333:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495851264 00000100:00100000:2.0:1789760755.355384:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739569808932 time 739619380861 state 0x0 ref 1 00000100:00100000:2.0:1789760755.355399:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739619641277 time 739619380861 state 0x1 ref 1 00000100:00100000:2.0:1789760755.355433:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739619641277 time 739676149920 state 0x0 ref 1 00000100:00100000:2.0:1789760755.355445:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495851264:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760755.355490:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24633344, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:2.0:1789760755.355499:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546c3e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.355515:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760755.355520:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760755.355526:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546c3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.355543:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546c3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760755.391342:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973331 is committed 00002000:00010000:2.0:1789760755.391440:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24633344 -> 24637440 00010000:00010000:2.0:1789760755.391455:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546c3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.391466:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546c3e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.391475:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546c3e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.391482:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760755.391487:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546c3e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.391495:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546c3e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.391512:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546c3e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.391520:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546c3e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.391606:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495851264:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36159us (36398us total) trans 4294973331 rc 0/0 00000100:00100000:2.0:1789760755.391615:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6043 00000100:00100000:2.0:1789760755.391623:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739619641277 time 739676149920 state 0x2 ref 0 00000100:00100000:1.0:1789760755.403203:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760755.403423:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495851392 00000100:00100000:3.0:1789760755.403433:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495851392 00000100:00100000:3.0:1789760755.403485:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739619641277 time 739676149920 state 0x0 ref 1 00000100:00100000:3.0:1789760755.403500:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739676410336 time 739676149920 state 0x1 ref 1 00000100:00100000:3.0:1789760755.403530:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739676410336 time 739724247406 state 0x0 ref 1 00000100:00100000:3.0:1789760755.403543:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495851392:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760755.403590:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24637440, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:3.0:1789760755.403599:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546c45 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.403614:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760755.403618:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.403624:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546c45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.403637:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546c45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760755.462227:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973332 is committed 00002000:00010000:0.0:1789760755.462320:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24637440 -> 24641536 00010000:00010000:0.0:1789760755.462335:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546c45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.462349:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546c45 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.462359:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546c45 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.462367:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760755.462373:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546c45 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.462382:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546c45 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.462406:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546c45 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.462413:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546c45 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760755.463112:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495851392:12345-192.168.203.9@tcp:4:dd.0 Request processed in 59565us (59911us total) trans 4294973332 rc 0/0 00000100:00100000:0.0:1789760755.463123:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6044 00000100:00100000:0.0:1789760755.463133:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739676410336 time 739724247406 state 0x2 ref 0 00000100:00100000:1.0:1789760755.467208:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760755.467323:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495851520 00000100:00100000:3.0:1789760755.467332:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495851520 00000100:00100000:3.0:1789760755.467380:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739676410336 time 739724247406 state 0x0 ref 1 00000100:00100000:3.0:1789760755.467392:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739724507822 time 739724247406 state 0x1 ref 1 00000100:00100000:3.0:1789760755.467413:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739724507822 time 739788130996 state 0x0 ref 1 00000100:00100000:3.0:1789760755.467422:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495851520:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760755.467473:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24641536, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:3.0:1789760755.467482:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546c4c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.467509:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760755.467514:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.467521:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546c4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.467535:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546c4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760755.514543:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973333 is committed 00002000:00010000:0.0:1789760755.514745:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24641536 -> 24645632 00010000:00010000:0.0:1789760755.514760:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546c4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.514773:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546c4c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.514782:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546c4c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.514790:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760755.514795:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546c4c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.514811:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546c4c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.514830:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546c4c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.514838:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546c4c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760755.514945:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495851520:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47512us (47733us total) trans 4294973333 rc 0/0 00000100:00100000:0.0:1789760755.514960:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6045 00000100:00100000:0.0:1789760755.514970:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739724507822 time 739788130996 state 0x2 ref 0 00000100:00100000:0.0:1789760755.518983:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760755.519201:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495851648 00000100:00100000:2.0:1789760755.519210:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495851648 00000100:00100000:2.0:1789760755.519260:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739724507822 time 739788130996 state 0x0 ref 1 00000100:00100000:2.0:1789760755.519275:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739788391412 time 739788130996 state 0x1 ref 1 00000100:00100000:2.0:1789760755.519608:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739788391412 time 739840323329 state 0x0 ref 1 00000100:00100000:2.0:1789760755.519622:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495851648:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760755.519666:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24645632, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:2.0:1789760755.519675:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546c53 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.519690:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760755.519694:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760755.519701:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546c53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.519714:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546c53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.550536:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760755.555725:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973334 is committed 00002000:00010000:2.0:1789760755.555806:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24645632 -> 24649728 00010000:00010000:2.0:1789760755.555821:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546c53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.555834:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546c53 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.555844:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546c53 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.555851:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760755.555856:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546c53 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.555865:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546c53 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.555883:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546c53 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.555893:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546c53 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.557304:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495851648:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37652us (38298us total) trans 4294973334 rc 0/0 00000100:00100000:2.0:1789760755.557322:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6046 00000100:00100000:2.0:1789760755.557334:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739788391412 time 739840323329 state 0x2 ref 0 00000100:00100000:2.0:1789760755.565129:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760755.565265:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495851776 00000100:00100000:0.0:1789760755.565275:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495851776 00000100:00100000:0.0:1789760755.565376:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739788391412 time 739840323329 state 0x0 ref 1 00000100:00100000:0.0:1789760755.565393:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739840583745 time 739840323329 state 0x1 ref 1 00000100:00100000:0.0:1789760755.565415:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739840583745 time 739886131830 state 0x0 ref 1 00000100:00100000:0.0:1789760755.565427:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495851776:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760755.565474:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24649728, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:0.0:1789760755.565484:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546c5a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.565501:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760755.565507:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.565513:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546c5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.565528:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546c5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760755.625334:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973335 is committed 00002000:00010000:2.0:1789760755.625453:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24649728 -> 24653824 00010000:00010000:2.0:1789760755.625468:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546c5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.625480:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546c5a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.625489:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546c5a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.625496:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760755.625501:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546c5a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.625510:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546c5a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.625528:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546c5a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.625536:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546c5a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.625630:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495851776:12345-192.168.203.9@tcp:4:dd.0 Request processed in 60202us (60508us total) trans 4294973335 rc 0/0 00000100:00100000:2.0:1789760755.625640:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6047 00000100:00100000:2.0:1789760755.625650:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739840583745 time 739886131830 state 0x2 ref 0 00000100:00100000:1.0:1789760755.633766:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760755.633878:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495851904 00000100:00100000:0.0:1789760755.633888:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495851904 00000100:00100000:0.0:1789760755.634015:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739840583745 time 739886131830 state 0x0 ref 1 00000100:00100000:0.0:1789760755.634035:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739886392246 time 739886131830 state 0x1 ref 1 00000100:00100000:0.0:1789760755.634102:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739886392246 time 739954817732 state 0x0 ref 1 00000100:00100000:0.0:1789760755.634116:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495851904:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760755.634162:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24653824, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:0.0:1789760755.634172:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546c61 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.634188:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760755.634192:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.634199:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546c61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.634212:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546c61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760755.665276:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24653824 -> 24657920 00002000:00010000:1.0:1789760755.665310:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760754 -> 1789760755 00002000:00010000:1.0:1789760755.665313:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760754 -> 1789760755 00010000:00010000:1.0:1789760755.665327:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546c61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.665340:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546c61 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.665349:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546c61 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.665357:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760755.665362:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546c61 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.665371:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546c61 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.665403:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546c61 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.665411:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546c61 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.665492:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495851904:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31375us (31731us total) trans 4294973336 rc 0/0 00000100:00100000:1.0:1789760755.665502:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6048 00000100:00100000:1.0:1789760755.665511:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739886392246 time 739954817732 state 0x2 ref 0 00000001:00080000:2.0:1789760755.665703:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973336 is committed 00000100:00100000:0.0:1789760755.674185:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760755.678537:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495852032 00000100:00100000:2.0:1789760755.678552:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495852032 00000100:00100000:0.0:1789760755.679207:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739886392246 time 739954817732 state 0x0 ref 1 00000100:00100000:0.0:1789760755.679223:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739955078148 time 739954817732 state 0x1 ref 1 00000100:00100000:0.0:1789760755.679257:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739955078148 time 739999974400 state 0x0 ref 1 00000100:00100000:0.0:1789760755.679270:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495852032:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760755.679346:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24657920, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc007 00010000:00010000:0.0:1789760755.679354:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546c68 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.679371:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760755.679375:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.679381:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546c68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.679459:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546c68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760755.730250:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973337 is committed 00002000:00010000:1.0:1789760755.731264:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24657920 -> 24662016 00010000:00010000:1.0:1789760755.731312:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546c68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.731469:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546c68 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.731487:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546c68 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.731495:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760755.731500:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546c68 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.731509:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546c68 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.731531:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546c68 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.731539:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546c68 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.731639:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495852032:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52366us (57457us total) trans 4294973337 rc 0/0 00000100:00100000:1.0:1789760755.731650:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6049 00000100:00100000:1.0:1789760755.731660:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739955078148 time 739999974400 state 0x2 ref 0 00000100:00100000:0.0:1789760755.738551:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760755.738732:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495852160 00000100:00100000:1.0:1789760755.738741:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495852160 00000100:00100000:1.0:1789760755.738792:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 739955078148 time 739999974400 state 0x0 ref 1 00000100:00100000:1.0:1789760755.738806:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740000234816 time 739999974400 state 0x1 ref 1 00000100:00100000:1.0:1789760755.738839:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740000234816 time 740059555283 state 0x0 ref 1 00000100:00100000:1.0:1789760755.738850:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495852160:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760755.738897:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24662016, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc007 00010000:00010000:1.0:1789760755.738906:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546c6f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.738930:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760755.738943:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760755.738950:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546c6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.738963:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546c6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.808365:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973338 is committed 00002000:00010000:1.0:1789760755.808524:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24662016 -> 24666112 00010000:00010000:1.0:1789760755.808540:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546c6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.808555:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546c6f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.808564:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546c6f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.808571:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760755.808576:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546c6f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.808584:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546c6f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.808603:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546c6f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.808610:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546c6f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.808741:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495852160:12345-192.168.203.9@tcp:4:dd.0 Request processed in 69886us (70193us total) trans 4294973338 rc 0/0 00000100:00100000:1.0:1789760755.808752:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6050 00000100:00100000:1.0:1789760755.808762:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740000234816 time 740059555283 state 0x2 ref 0 00000100:00100000:2.0:1789760755.820080:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760755.820227:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495852288 00000100:00100000:1.0:1789760755.820236:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495852288 00000100:00100000:1.0:1789760755.820325:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740000234816 time 740059555283 state 0x0 ref 1 00000100:00100000:1.0:1789760755.820341:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740059815699 time 740059555283 state 0x1 ref 1 00000100:00100000:1.0:1789760755.820371:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740059815699 time 740141088298 state 0x0 ref 1 00000100:00100000:1.0:1789760755.820384:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495852288:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760755.820433:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24666112, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc007 00010000:00010000:1.0:1789760755.820444:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546c76 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.820460:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760755.820464:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760755.820470:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546c76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.820484:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546c76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760755.884760:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973339 is committed 00002000:00010000:0.0:1789760755.884854:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24666112 -> 24670208 00010000:00010000:0.0:1789760755.884870:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546c76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.884883:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546c76 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.884891:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546c76 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.884898:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760755.884903:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546c76 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.884911:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546c76 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.885251:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546c76 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.885262:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546c76 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760755.885813:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495852288:12345-192.168.203.9@tcp:4:dd.0 Request processed in 65426us (65736us total) trans 4294973339 rc 0/0 00000100:00100000:0.0:1789760755.885823:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6051 00000100:00100000:0.0:1789760755.885833:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740059815699 time 740141088298 state 0x2 ref 0 00000100:00100000:3.0:1789760755.892540:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760755.892823:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495852416 00000100:00100000:1.0:1789760755.892837:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495852416 00000100:00100000:1.0:1789760755.893011:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740059815699 time 740141088298 state 0x0 ref 1 00000100:00100000:1.0:1789760755.893032:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740141348714 time 740141088298 state 0x1 ref 1 00000100:00100000:1.0:1789760755.893070:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740141348714 time 740213785434 state 0x0 ref 1 00000100:00100000:1.0:1789760755.893085:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495852416:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760755.893159:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24670208, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc007 00010000:00010000:1.0:1789760755.893171:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546c7d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.893197:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760755.893203:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760755.893510:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546c7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.893534:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546c7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.968852:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973340 is committed 00002000:00010000:0.0:1789760755.969060:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24670208 -> 24674304 00010000:00010000:0.0:1789760755.969081:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546c7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.969095:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546c7d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.969104:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546c7d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.969112:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760755.969117:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546c7d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.969127:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546c7d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.969151:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546c7d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.969159:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546c7d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760755.969366:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495852416:12345-192.168.203.9@tcp:4:dd.0 Request processed in 76277us (76837us total) trans 4294973340 rc 0/0 00000100:00100000:0.0:1789760755.969379:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6052 00000100:00100000:0.0:1789760755.969393:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740141348714 time 740213785434 state 0x2 ref 0 00000100:00100000:2.0:1789760755.974415:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760755.977195:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495852544 00000100:00100000:1.0:1789760755.977215:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495852544 00000100:00100000:0.0:1789760755.980379:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740141348714 time 740213785434 state 0x0 ref 1 00000100:00100000:0.0:1789760755.980403:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740214045850 time 740213785434 state 0x1 ref 1 00000100:00100000:0.0:1789760755.980456:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740214045850 time 740301171753 state 0x0 ref 1 00000100:00100000:0.0:1789760755.980474:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495852544:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760755.980545:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24674304, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc007 00010000:00010000:0.0:1789760755.980556:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546c84 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.980575:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760755.980580:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.980588:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546c84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.980605:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546c84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760756.000731:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff92900fab0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520310016:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760756.000841:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760756.001130:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520310016 00000100:00100000:2.0:1789760756.001142:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520310016 00000100:00080000:3.0:1789760756.001214:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760756.001218:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 205 00000100:00100000:2.0:1789760756.001230:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520310016:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760756.001300:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528154624 pending 1703936 free 3833593856 avail 3808428032 00000100:00100000:2.0:1789760756.001357:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520310016:12345-0@lo:13:osp-pre-0-0.0 Request processed in 128us (514us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760756.001365:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 205 00000100:00100000:0.0:1789760756.001767:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff92900fab0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520310016:0@lo:13:osp-pre-0-0.0 00000001:00080000:3.0:1789760756.027664:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973341 is committed 00002000:00010000:2.0:1789760756.027797:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24674304 -> 24678400 00010000:00010000:2.0:1789760756.027818:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546c84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.027833:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546c84 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.027850:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546c84 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.027858:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760756.027865:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546c84 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.027875:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546c84 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.027901:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546c84 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.027926:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546c84 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.028076:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495852544:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47600us (53668us total) trans 4294973341 rc 0/0 00000100:00100000:2.0:1789760756.028088:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6053 00000100:00100000:2.0:1789760756.028101:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740214045850 time 740301171753 state 0x2 ref 0 00000100:00100000:2.0:1789760756.036910:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760756.037115:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495852672 00000100:00100000:0.0:1789760756.037130:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495852672 00000100:00100000:0.0:1789760756.037208:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740214045850 time 740301171753 state 0x0 ref 1 00000100:00100000:0.0:1789760756.037228:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740301432169 time 740301171753 state 0x1 ref 1 00000100:00100000:0.0:1789760756.037271:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740301432169 time 740357986834 state 0x0 ref 1 00000100:00100000:0.0:1789760756.037335:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495852672:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760756.037402:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24678400, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc007 00010000:00010000:0.0:1789760756.037413:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546c8b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.037432:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760756.037437:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760756.037446:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546c8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.037462:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546c8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760756.117188:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973342 is committed 00002000:00010000:2.0:1789760756.117439:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24678400 -> 24682496 00010000:00010000:2.0:1789760756.117462:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546c8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.117483:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546c8b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.117494:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546c8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.117501:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760756.117507:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546c8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.117517:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546c8b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.117542:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546c8b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.117551:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546c8b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.117718:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495852672:12345-192.168.203.9@tcp:4:dd.0 Request processed in 80380us (80815us total) trans 4294973342 rc 0/0 00000100:00100000:2.0:1789760756.117731:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6054 00000100:00100000:2.0:1789760756.117744:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740301432169 time 740357986834 state 0x2 ref 0 00000100:00100000:3.0:1789760756.120531:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760756.120685:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495852800 00000100:00100000:1.0:1789760756.120696:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495852800 00000100:00100000:1.0:1789760756.120770:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740301432169 time 740357986834 state 0x0 ref 1 00000100:00100000:1.0:1789760756.120788:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740358247250 time 740357986834 state 0x1 ref 1 00000100:00100000:1.0:1789760756.120822:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740358247250 time 740441538484 state 0x0 ref 1 00000100:00100000:1.0:1789760756.120836:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495852800:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760756.120895:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24682496, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc007 00010000:00010000:1.0:1789760756.120905:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546c92 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.120944:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760756.120950:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.120957:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546c92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.120973:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546c92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760756.172839:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973343 is committed 00002000:00010000:2.0:1789760756.172869:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24682496 -> 24686592 00010000:00010000:2.0:1789760756.172888:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546c92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.172904:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546c92 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.172924:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546c92 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.172942:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760756.172948:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546c92 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.172958:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546c92 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.172981:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546c92 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.172991:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354546c92 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.173114:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495852800:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52274us (52589us total) trans 4294973343 rc 0/0 00000100:00100000:2.0:1789760756.173126:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6055 00000100:00100000:2.0:1789760756.173139:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740358247250 time 740441538484 state 0x2 ref 0 00000100:00100000:0.0:1789760756.179718:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760756.184112:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495852928 00000100:00100000:2.0:1789760756.184134:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495852928 00000100:00100000:2.0:1789760756.184225:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740358247250 time 740441538484 state 0x0 ref 1 00000100:00100000:2.0:1789760756.184246:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740441798900 time 740441538484 state 0x1 ref 1 00000100:00100000:2.0:1789760756.184306:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740441798900 time 740505022116 state 0x0 ref 1 00000100:00100000:2.0:1789760756.184322:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495852928:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760756.184390:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24686592, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc007 00010000:00010000:2.0:1789760756.184400:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546c99 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.184420:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760756.184432:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.184440:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546c99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.184457:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546c99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760756.224501:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973344 is committed 00002000:00010000:2.0:1789760756.224610:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24686592 -> 24690688 00010000:00010000:2.0:1789760756.224630:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546c99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.224646:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546c99 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.224658:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546c99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.224666:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760756.224673:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546c99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.224684:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546c99 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.224708:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546c99 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.224717:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546c99 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.224878:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495852928:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40553us (45168us total) trans 4294973344 rc 0/0 00000100:00100000:2.0:1789760756.224890:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6056 00000100:00100000:2.0:1789760756.224903:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740441798900 time 740505022116 state 0x2 ref 0 00000100:00100000:1.0:1789760756.227947:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760756.228045:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495853056 00000100:00100000:0.0:1789760756.228058:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495853056 00000100:00100000:0.0:1789760756.228136:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740441798900 time 740505022116 state 0x0 ref 1 00000100:00100000:0.0:1789760756.228156:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740505282532 time 740505022116 state 0x1 ref 1 00000100:00100000:0.0:1789760756.228190:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740505282532 time 740548907031 state 0x0 ref 1 00000100:00100000:0.0:1789760756.228207:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495853056:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760756.228307:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24690688, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc007 00010000:00010000:0.0:1789760756.228319:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546ca0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.228340:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760756.228345:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760756.228353:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546ca0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.228369:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546ca0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760756.259701:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973345 is committed 00002000:00010000:2.0:1789760756.259841:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24690688 -> 24694784 00010000:00010000:2.0:1789760756.259862:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546ca0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.259885:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546ca0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.259895:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546ca0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.259903:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760756.259919:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546ca0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.259940:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546ca0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.259967:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546ca0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.259977:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354546ca0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.260094:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495853056:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31884us (32165us total) trans 4294973345 rc 0/0 00000100:00100000:2.0:1789760756.260106:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6057 00000100:00100000:2.0:1789760756.260118:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740505282532 time 740548907031 state 0x2 ref 0 00000100:00100000:1.0:1789760756.263608:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760756.263713:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495853184 00000100:00100000:2.0:1789760756.263727:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495853184 00000100:00100000:2.0:1789760756.263791:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740505282532 time 740548907031 state 0x0 ref 1 00000100:00100000:2.0:1789760756.263808:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740549167447 time 740548907031 state 0x1 ref 1 00000100:00100000:2.0:1789760756.263872:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740549167447 time 740584588696 state 0x0 ref 1 00000100:00100000:2.0:1789760756.263886:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495853184:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760756.263968:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24694784, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc007 00010000:00010000:2.0:1789760756.263980:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546ca7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.263999:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760756.264005:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.264012:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546ca7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.264028:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546ca7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760756.294791:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24694784 -> 24698880 00010000:00010000:2.0:1789760756.294820:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546ca7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.294837:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546ca7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.294853:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546ca7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.294861:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760756.294867:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546ca7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.294878:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546ca7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.294904:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546ca7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.295003:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546ca7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.295170:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495853184:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31278us (31568us total) trans 4294973346 rc 0/0 00000100:00100000:2.0:1789760756.295185:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6058 00000100:00100000:2.0:1789760756.295198:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740549167447 time 740584588696 state 0x2 ref 0 00000001:00080000:3.0:1789760756.295515:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973346 is committed 00000100:00100000:3.0:1789760756.298891:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760756.299065:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495853312 00000100:00100000:2.0:1789760756.299078:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495853312 00000100:00100000:2.0:1789760756.299149:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740549167447 time 740584588696 state 0x0 ref 1 00000100:00100000:2.0:1789760756.299167:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740584849112 time 740584588696 state 0x1 ref 1 00000100:00100000:2.0:1789760756.299230:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740584849112 time 740619947216 state 0x0 ref 1 00000100:00100000:2.0:1789760756.299243:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495853312:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760756.299425:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24698880, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc007 00010000:00010000:2.0:1789760756.299437:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.299457:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760756.299462:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.299470:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.299488:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760756.332608:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24698880 -> 24702976 00010000:00010000:2.0:1789760756.332634:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.332650:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.332660:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.332669:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760756.332674:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.332685:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.332709:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.332718:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.332836:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495853312:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33587us (33952us total) trans 4294973347 rc 0/0 00000100:00100000:2.0:1789760756.332850:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6059 00000100:00100000:2.0:1789760756.332863:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740584849112 time 740619947216 state 0x2 ref 0 00000001:00080000:3.0:1789760756.333021:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973347 is committed 00000100:00100000:2.0:1789760756.336222:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760756.336648:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495853440 00000100:00100000:0.0:1789760756.336662:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495853440 00000100:00100000:0.0:1789760756.336737:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740584849112 time 740619947216 state 0x0 ref 1 00000100:00100000:0.0:1789760756.336758:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740620207632 time 740619947216 state 0x1 ref 1 00000100:00100000:0.0:1789760756.336793:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740620207632 time 740657509829 state 0x0 ref 1 00000100:00100000:0.0:1789760756.336808:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495853440:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760756.336872:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24702976, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc007 00010000:00010000:0.0:1789760756.336882:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546cb5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.336918:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760756.336923:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760756.336939:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546cb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.336956:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546cb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760756.377887:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973348 is committed 00002000:00010000:1.0:1789760756.378120:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24702976 -> 24707072 00010000:00010000:1.0:1789760756.378143:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546cb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.378158:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546cb5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.378168:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546cb5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.378176:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760756.378182:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546cb5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.378192:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546cb5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.378217:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546cb5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.378226:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546cb5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760756.378582:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495853440:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41769us (42366us total) trans 4294973348 rc 0/0 00000100:00100000:1.0:1789760756.378595:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6060 00000100:00100000:1.0:1789760756.378608:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740620207632 time 740657509829 state 0x2 ref 0 00000100:00100000:0.0:1789760756.388952:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760756.389098:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495853568 00000100:00100000:2.0:1789760756.389112:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495853568 00000100:00100000:2.0:1789760756.389187:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740620207632 time 740657509829 state 0x0 ref 1 00000100:00100000:2.0:1789760756.389205:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740657770245 time 740657509829 state 0x1 ref 1 00000100:00100000:2.0:1789760756.389241:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740657770245 time 740709958199 state 0x0 ref 1 00000100:00100000:2.0:1789760756.389256:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495853568:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760756.389375:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24707072, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc007 00010000:00010000:2.0:1789760756.389387:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cbc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.389407:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760756.389412:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.389420:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cbc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.389438:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cbc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760756.519011:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24707072 -> 24711168 00010000:00010000:2.0:1789760756.519030:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cbc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.519043:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cbc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.519051:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cbc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.519058:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760756.519063:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cbc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.519072:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cbc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.519091:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cbc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.519099:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cbc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760756.519143:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973349 is committed 00000100:00100000:2.0:1789760756.519196:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495853568:12345-192.168.203.9@tcp:4:dd.0 Request processed in 129937us (130262us total) trans 4294973349 rc 0/0 00000100:00100000:2.0:1789760756.519207:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6061 00000100:00100000:2.0:1789760756.519217:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740657770245 time 740709958199 state 0x2 ref 0 00000100:00100000:3.0:1789760756.524458:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760756.524796:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495853696 00000100:00100000:2.0:1789760756.524810:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495853696 00000100:00100000:2.0:1789760756.524885:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740657770245 time 740709958199 state 0x0 ref 1 00000100:00100000:2.0:1789760756.524905:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740710218615 time 740709958199 state 0x1 ref 1 00000100:00100000:2.0:1789760756.525086:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740710218615 time 740845800958 state 0x0 ref 1 00000100:00100000:2.0:1789760756.525101:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495853696:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760756.525168:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24711168, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc007 00010000:00010000:2.0:1789760756.525179:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cc3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.525198:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760756.525204:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.525211:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cc3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.525228:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cc3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760756.571623:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973350 is committed 00002000:00010000:2.0:1789760756.571755:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24711168 -> 24715264 00010000:00010000:2.0:1789760756.571775:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cc3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.571790:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cc3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.571800:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cc3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.571809:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760756.571815:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cc3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.571825:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cc3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.571849:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cc3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.571862:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cc3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.572792:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495853696:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47679us (48335us total) trans 4294973350 rc 0/0 00000100:00100000:2.0:1789760756.572806:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6062 00000100:00100000:2.0:1789760756.572820:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740710218615 time 740845800958 state 0x2 ref 0 00010000:00010000:0.0:1789760756.574556:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760756.580212:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760756.580711:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495853824 00000100:00100000:1.0:1789760756.580724:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495853824 00000100:00100000:1.0:1789760756.580803:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740710218615 time 740845800958 state 0x0 ref 1 00000100:00100000:1.0:1789760756.580823:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740846061374 time 740845800958 state 0x1 ref 1 00000100:00100000:1.0:1789760756.582031:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740846061374 time 740902739161 state 0x0 ref 1 00000100:00100000:1.0:1789760756.582067:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495853824:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760756.582143:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24715264, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc007 00010000:00010000:1.0:1789760756.582155:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546cca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.582175:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760756.582181:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.582188:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546cca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.582206:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546cca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760756.659123:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973351 is committed 00002000:00010000:1.0:1789760756.659997:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24715264 -> 24719360 00010000:00010000:1.0:1789760756.660017:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546cca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.660033:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546cca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.660045:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546cca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.660054:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760756.660060:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546cca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.660071:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546cca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.660099:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546cca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.660108:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546cca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760756.660235:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495853824:12345-192.168.203.9@tcp:4:dd.0 Request processed in 78171us (80029us total) trans 4294973351 rc 0/0 00000100:00100000:1.0:1789760756.660253:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6063 00000100:00100000:1.0:1789760756.660266:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740846061374 time 740902739161 state 0x2 ref 0 00000100:00100000:1.0:1789760756.676435:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760756.676559:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495853952 00000100:00100000:2.0:1789760756.676574:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495853952 00000100:00100000:2.0:1789760756.676651:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740846061374 time 740902739161 state 0x0 ref 1 00000100:00100000:2.0:1789760756.676672:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740902999577 time 740902739161 state 0x1 ref 1 00000100:00100000:2.0:1789760756.676709:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740902999577 time 740997424450 state 0x0 ref 1 00000100:00100000:2.0:1789760756.676725:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495853952:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760756.676792:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24719360, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc007 00010000:00010000:2.0:1789760756.676803:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cd1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.676824:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760756.676829:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.676837:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cd1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.676855:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cd1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760756.744463:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24719360 -> 24723456 00002000:00010000:1.0:1789760756.744472:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760755 -> 1789760756 00002000:00010000:1.0:1789760756.744475:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760755 -> 1789760756 00010000:00010000:1.0:1789760756.744489:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cd1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.744502:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cd1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.744511:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cd1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.744518:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760756.744523:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cd1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.744532:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cd1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.744553:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cd1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.744567:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546cd1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760756.744660:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495853952:12345-192.168.203.9@tcp:4:dd.0 Request processed in 67934us (68236us total) trans 4294973352 rc 0/0 00000100:00100000:1.0:1789760756.744672:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6064 00000100:00100000:1.0:1789760756.744682:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740902999577 time 740997424450 state 0x2 ref 0 00000001:00080000:2.0:1789760756.745394:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973352 is committed 00000100:00100000:3.0:1789760756.747702:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760756.748095:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495854080 00000100:00100000:2.0:1789760756.748110:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495854080 00000100:00100000:2.0:1789760756.748175:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740902999577 time 740997424450 state 0x0 ref 1 00000100:00100000:2.0:1789760756.748190:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740997684866 time 740997424450 state 0x1 ref 1 00000100:00100000:2.0:1789760756.748224:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740997684866 time 741068941177 state 0x0 ref 1 00000100:00100000:2.0:1789760756.748242:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495854080:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760756.749142:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24723456, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc007 00010000:00010000:2.0:1789760756.749154:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546cd8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.749171:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760756.749176:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.749182:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546cd8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.749197:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546cd8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760756.828914:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973353 is committed 00002000:00010000:0.0:1789760756.833362:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24723456 -> 24727552 00010000:00010000:0.0:1789760756.833383:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546cd8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.833397:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546cd8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.833407:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546cd8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.833415:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760756.833420:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546cd8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.833437:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546cd8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.833460:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546cd8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.833468:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546cd8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760756.833759:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495854080:12345-192.168.203.9@tcp:4:dd.0 Request processed in 85518us (86068us total) trans 4294973353 rc 0/0 00000100:00100000:0.0:1789760756.833770:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6065 00000100:00100000:0.0:1789760756.833781:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740997684866 time 741068941177 state 0x2 ref 0 00000100:00100000:3.0:1789760756.837437:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760756.837669:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495854208 00000100:00100000:1.0:1789760756.837684:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495854208 00000100:00100000:1.0:1789760756.837745:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 740997684866 time 741068941177 state 0x0 ref 1 00000100:00100000:1.0:1789760756.837761:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741069201593 time 741068941177 state 0x1 ref 1 00000100:00100000:1.0:1789760756.837794:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741069201593 time 741158511105 state 0x0 ref 1 00000100:00100000:1.0:1789760756.837810:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495854208:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760756.837861:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24727552, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc007 00010000:00010000:1.0:1789760756.837871:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546cdf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.837888:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760756.837894:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.837901:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546cdf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.837940:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546cdf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760756.926891:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24727552 -> 24731648 00010000:00010000:1.0:1789760756.926935:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546cdf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.926952:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546cdf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.926963:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546cdf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.927096:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760756.927103:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546cdf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.927116:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546cdf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.927145:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546cdf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.927155:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546cdf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760756.927346:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495854208:12345-192.168.203.9@tcp:4:dd.0 Request processed in 89532us (89917us total) trans 4294973354 rc 0/0 00000100:00100000:1.0:1789760756.927359:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6066 00000100:00100000:1.0:1789760756.927372:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741069201593 time 741158511105 state 0x2 ref 0 00000001:00080000:3.0:1789760756.930706:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973354 is committed 00000100:00100000:1.0:1789760756.930906:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760756.931128:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495854336 00000100:00100000:3.0:1789760756.931142:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495854336 00000100:00100000:3.0:1789760756.931221:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741069201593 time 741158511105 state 0x0 ref 1 00000100:00100000:3.0:1789760756.931241:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741158771521 time 741158511105 state 0x1 ref 1 00000100:00100000:3.0:1789760756.931329:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741158771521 time 741252044671 state 0x0 ref 1 00000100:00100000:3.0:1789760756.931346:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903639b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495854336:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760756.931415:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24731648, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc007 00010000:00010000:3.0:1789760756.931426:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ce6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.931449:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760756.931454:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.931462:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ce6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.931479:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ce6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760756.989198:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973355 is committed 00002000:00010000:0.0:1789760756.989467:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24731648 -> 24735744 00010000:00010000:0.0:1789760756.989488:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ce6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.989505:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ce6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.989516:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ce6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.989525:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760756.989532:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ce6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.989543:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ce6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.989569:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ce6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.989577:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ce6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760756.989704:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903639b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495854336:12345-192.168.203.9@tcp:4:dd.0 Request processed in 58353us (58796us total) trans 4294973355 rc 0/0 00000100:00100000:0.0:1789760756.989717:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6067 00000100:00100000:0.0:1789760756.989729:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741158771521 time 741252044671 state 0x2 ref 0 00000100:00100000:2.0:1789760756.993740:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760756.993858:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495854464 00000100:00100000:0.0:1789760756.993871:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495854464 00000100:00100000:0.0:1789760756.993968:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741158771521 time 741252044671 state 0x0 ref 1 00000100:00100000:0.0:1789760756.993986:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741252305087 time 741252044671 state 0x1 ref 1 00000100:00100000:0.0:1789760756.994070:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741252305087 time 741314731015 state 0x0 ref 1 00000100:00100000:0.0:1789760756.994085:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495854464:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760756.994144:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24735744, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc007 00010000:00010000:0.0:1789760756.994155:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546ced lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.994174:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760756.994179:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760756.994186:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546ced lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.994202:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546ced lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760757.028885:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973356 is committed 00002000:00010000:0.0:1789760757.029043:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24735744 -> 24739840 00010000:00010000:0.0:1789760757.029063:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546ced lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.029078:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546ced lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.029088:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546ced lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.029097:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760757.029103:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546ced lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.029113:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546ced lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.029145:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546ced lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.029153:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546ced lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.029388:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+6:2357:x1876699495854464:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35298us (35657us total) trans 4294973356 rc 0/0 00000100:00100000:0.0:1789760757.029401:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6068 00000100:00100000:0.0:1789760757.029413:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741252305087 time 741314731015 state 0x2 ref 0 00000100:00100000:2.0:1789760757.033340:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760757.033491:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495854592 00000100:00100000:1.0:1789760757.033504:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495854592 00000100:00100000:1.0:1789760757.033597:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741252305087 time 741314731015 state 0x0 ref 1 00000100:00100000:1.0:1789760757.033618:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741314991431 time 741314731015 state 0x1 ref 1 00000100:00100000:1.0:1789760757.033658:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741314991431 time 741354374042 state 0x0 ref 1 00000100:00100000:1.0:1789760757.033674:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495854592:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760757.033738:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24739840, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc007 00010000:00010000:1.0:1789760757.033749:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546cf4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.033769:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760757.033774:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.033782:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546cf4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.033798:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546cf4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760757.034609:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760757.068561:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973357 is committed 00002000:00010000:0.0:1789760757.069154:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24739840 -> 24743936 00010000:00010000:0.0:1789760757.069177:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546cf4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.069193:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546cf4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.069203:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546cf4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.069211:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760757.069217:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546cf4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.069232:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546cf4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.069271:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546cf4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.069307:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546cf4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.069440:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495854592:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35763us (36108us total) trans 4294973357 rc 0/0 00000100:00100000:0.0:1789760757.069453:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6069 00000100:00100000:0.0:1789760757.069466:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741314991431 time 741354374042 state 0x2 ref 0 00000100:00100000:0.0:1789760757.072951:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760757.073110:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495854720 00000100:00100000:3.0:1789760757.073124:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495854720 00000100:00100000:3.0:1789760757.073196:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741314991431 time 741354374042 state 0x0 ref 1 00000100:00100000:3.0:1789760757.073217:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741354634458 time 741354374042 state 0x1 ref 1 00000100:00100000:3.0:1789760757.073245:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741354634458 time 741393962443 state 0x0 ref 1 00000100:00100000:3.0:1789760757.073270:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495854720:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760757.073371:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24743936, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc007 00010000:00010000:3.0:1789760757.073385:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546cfb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.073405:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760757.073411:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.073418:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546cfb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.073435:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546cfb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760757.102107:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973358 is committed 00002000:00010000:0.0:1789760757.103273:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24743936 -> 24748032 00010000:00010000:0.0:1789760757.103336:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546cfb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.103352:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546cfb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.103363:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546cfb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.103376:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760757.103382:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546cfb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.103392:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546cfb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.103418:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546cfb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.103427:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546cfb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.103570:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495854720:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30295us (30624us total) trans 4294973358 rc 0/0 00000100:00100000:0.0:1789760757.103582:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6070 00000100:00100000:0.0:1789760757.103596:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741354634458 time 741393962443 state 0x2 ref 0 00000100:00100000:2.0:1789760757.113548:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760757.113656:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495854848 00000100:00100000:0.0:1789760757.113670:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495854848 00000100:00100000:0.0:1789760757.113741:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741354634458 time 741393962443 state 0x0 ref 1 00000100:00100000:0.0:1789760757.113760:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741394222859 time 741393962443 state 0x1 ref 1 00000100:00100000:0.0:1789760757.113789:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741394222859 time 741434505122 state 0x0 ref 1 00000100:00100000:0.0:1789760757.113805:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495854848:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760757.113868:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24748032, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc007 00010000:00010000:0.0:1789760757.113879:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546d02 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.113899:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760757.113915:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.113932:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546d02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.113949:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546d02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760757.164405:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973359 is committed 00002000:00010000:1.0:1789760757.164542:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24748032 -> 24752128 00010000:00010000:1.0:1789760757.164563:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546d02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.164580:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546d02 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.164590:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546d02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.164598:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760757.164604:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546d02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.164614:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546d02 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.164639:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546d02 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.164647:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354546d02 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.164773:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495854848:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50963us (51233us total) trans 4294973359 rc 0/0 00000100:00100000:1.0:1789760757.164786:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6071 00000100:00100000:1.0:1789760757.164799:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741394222859 time 741434505122 state 0x2 ref 0 00000100:00100000:3.0:1789760757.173249:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760757.173869:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495854976 00000100:00100000:0.0:1789760757.173884:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495854976 00000100:00100000:0.0:1789760757.174039:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741394222859 time 741434505122 state 0x0 ref 1 00000100:00100000:0.0:1789760757.174060:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741434765538 time 741434505122 state 0x1 ref 1 00000100:00100000:0.0:1789760757.174151:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741434765538 time 741494866625 state 0x0 ref 1 00000100:00100000:0.0:1789760757.174168:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495854976:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760757.174237:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24752128, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc007 00010000:00010000:0.0:1789760757.174250:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d09 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.174316:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760757.174323:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.174386:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.174407:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.218867:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973360 is committed 00002000:00010000:0.0:1789760757.219124:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24752128 -> 24756224 00010000:00010000:0.0:1789760757.219147:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.219163:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d09 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.219173:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.219182:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760757.219188:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.219198:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d09 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.219224:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d09 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.219234:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d09 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.219639:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495854976:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45467us (46397us total) trans 4294973360 rc 0/0 00000100:00100000:0.0:1789760757.219653:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6072 00000100:00100000:0.0:1789760757.219666:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741434765538 time 741494866625 state 0x2 ref 0 00000100:00100000:2.0:1789760757.223789:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760757.223991:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495855104 00000100:00100000:0.0:1789760757.224004:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495855104 00000100:00100000:0.0:1789760757.224077:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741434765538 time 741494866625 state 0x0 ref 1 00000100:00100000:0.0:1789760757.224096:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741495127041 time 741494866625 state 0x1 ref 1 00000100:00100000:0.0:1789760757.224133:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741495127041 time 741544848665 state 0x0 ref 1 00000100:00100000:0.0:1789760757.224148:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495855104:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760757.224211:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24756224, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc007 00010000:00010000:0.0:1789760757.224223:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d10 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.224255:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760757.224261:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.224269:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.224641:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.275521:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973361 is committed 00002000:00010000:0.0:1789760757.275750:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24756224 -> 24760320 00010000:00010000:0.0:1789760757.275771:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.275787:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d10 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.275797:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d10 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.275806:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760757.275812:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d10 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.275823:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d10 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.275848:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d10 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.275858:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d10 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.276020:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495855104:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51866us (52238us total) trans 4294973361 rc 0/0 00000100:00100000:0.0:1789760757.276035:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6073 00000100:00100000:0.0:1789760757.276051:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741495127041 time 741544848665 state 0x2 ref 0 00000100:00100000:3.0:1789760757.281015:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760757.281390:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495855232 00000100:00100000:1.0:1789760757.281405:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495855232 00000100:00100000:1.0:1789760757.281481:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741495127041 time 741544848665 state 0x0 ref 1 00000100:00100000:1.0:1789760757.281501:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741545109081 time 741544848665 state 0x1 ref 1 00000100:00100000:1.0:1789760757.281542:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741545109081 time 741602258399 state 0x0 ref 1 00000100:00100000:1.0:1789760757.281558:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495855232:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760757.281625:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24760320, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc007 00010000:00010000:1.0:1789760757.281638:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546d17 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.281659:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760757.281665:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.281673:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546d17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.281691:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546d17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760757.310729:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973362 is committed 00002000:00010000:0.0:1789760757.311311:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24760320 -> 24764416 00010000:00010000:0.0:1789760757.311335:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546d17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.311353:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546d17 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.311367:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546d17 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.311378:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760757.311384:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546d17 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.311395:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546d17 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.311425:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546d17 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.311443:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546d17 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.311683:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495855232:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30120us (30673us total) trans 4294973362 rc 0/0 00000100:00100000:0.0:1789760757.311696:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6074 00000100:00100000:0.0:1789760757.311710:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741545109081 time 741602258399 state 0x2 ref 0 00000100:00100000:3.0:1789760757.318407:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760757.318685:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495855360 00000100:00100000:0.0:1789760757.318698:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495855360 00000100:00100000:0.0:1789760757.318771:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741545109081 time 741602258399 state 0x0 ref 1 00000100:00100000:0.0:1789760757.318790:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741602518815 time 741602258399 state 0x1 ref 1 00000100:00100000:0.0:1789760757.318825:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741602518815 time 741639542666 state 0x0 ref 1 00000100:00100000:0.0:1789760757.318838:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495855360:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760757.318904:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24764416, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc007 00010000:00010000:0.0:1789760757.318916:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d1e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.318948:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760757.318954:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.318961:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.318982:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.359140:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973363 is committed 00002000:00010000:2.0:1789760757.359350:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24764416 -> 24768512 00010000:00010000:2.0:1789760757.359372:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.359389:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d1e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.359398:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d1e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.359406:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760757.359412:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d1e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.359422:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d1e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.359444:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d1e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.359452:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546d1e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.359567:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495855360:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40724us (41169us total) trans 4294973363 rc 0/0 00000100:00100000:2.0:1789760757.359580:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6075 00000100:00100000:2.0:1789760757.359593:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741602518815 time 741639542666 state 0x2 ref 0 00000100:00100000:2.0:1789760757.379018:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760757.379140:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495855488 00000100:00100000:0.0:1789760757.379153:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495855488 00000100:00100000:0.0:1789760757.379237:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741602518815 time 741639542666 state 0x0 ref 1 00000100:00100000:0.0:1789760757.379265:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741639803082 time 741639542666 state 0x1 ref 1 00000100:00100000:3.0:1789760757.379333:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741639803082 time 741700045416 state 0x0 ref 1 00000100:00100000:3.0:1789760757.379352:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495855488:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760757.379418:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24768512, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc007 00010000:00010000:3.0:1789760757.379429:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546d25 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.379448:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760757.379453:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.379461:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546d25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.379478:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546d25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760757.438732:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24768512 -> 24772608 00010000:00010000:3.0:1789760757.438766:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546d25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.438791:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546d25 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.438802:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546d25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.438819:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760757.438825:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546d25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.438836:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546d25 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.438885:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546d25 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.438895:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546d25 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.439488:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495855488:12345-192.168.203.9@tcp:4:dd.0 Request processed in 60132us (60475us total) trans 4294973364 rc 0/0 00000100:00100000:3.0:1789760757.439503:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6076 00000100:00100000:3.0:1789760757.439516:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741639803082 time 741700045416 state 0x2 ref 0 00000001:00080000:0.0:1789760757.440059:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973364 is committed 00000100:00100000:2.0:1789760757.444240:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760757.444683:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495855616 00000100:00100000:0.0:1789760757.444698:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495855616 00000100:00100000:0.0:1789760757.444775:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741639803082 time 741700045416 state 0x0 ref 1 00000100:00100000:0.0:1789760757.444795:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741700305832 time 741700045416 state 0x1 ref 1 00000100:00100000:0.0:1789760757.444831:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741700305832 time 741765546936 state 0x0 ref 1 00000100:00100000:0.0:1789760757.444845:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495855616:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760757.444910:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24772608, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc007 00010000:00010000:0.0:1789760757.444929:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546d2c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.444951:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760757.444956:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.444964:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546d2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.444982:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546d2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760757.523541:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24772608 -> 24776704 00010000:00010000:1.0:1789760757.523566:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546d2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.523582:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546d2c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.523591:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546d2c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.523600:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760757.523606:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546d2c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.523617:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546d2c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.523642:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546d2c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.523650:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546d2c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.523786:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495855616:12345-192.168.203.9@tcp:4:dd.0 Request processed in 78935us (79553us total) trans 4294973365 rc 0/0 00000100:00100000:1.0:1789760757.523799:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6077 00000100:00100000:1.0:1789760757.523812:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741700305832 time 741765546936 state 0x2 ref 0 00000001:00080000:2.0:1789760757.525965:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973365 is committed 00000100:00100000:2.0:1789760757.528129:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760757.528643:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495855744 00000100:00100000:0.0:1789760757.528657:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495855744 00000100:00100000:0.0:1789760757.528733:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741700305832 time 741765546936 state 0x0 ref 1 00000100:00100000:0.0:1789760757.528751:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741765807352 time 741765546936 state 0x1 ref 1 00000100:00100000:0.0:1789760757.528786:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741765807352 time 741849502225 state 0x0 ref 1 00000100:00100000:0.0:1789760757.528801:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495855744:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760757.528945:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24776704, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc007 00010000:00010000:0.0:1789760757.528957:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546d33 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.528977:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760757.528981:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.528988:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546d33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.529004:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546d33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.599222:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760757.604707:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973366 is committed 00002000:00010000:1.0:1789760757.616403:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24776704 -> 24780800 00010000:00010000:1.0:1789760757.616424:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546d33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.617111:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546d33 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.617122:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546d33 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.617130:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760757.617135:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546d33 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.617146:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546d33 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.617167:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546d33 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.617175:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546d33 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.617405:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495855744:12345-192.168.203.9@tcp:4:dd.0 Request processed in 88601us (89285us total) trans 4294973366 rc 0/0 00000100:00100000:1.0:1789760757.617418:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6078 00000100:00100000:1.0:1789760757.617430:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741765807352 time 741849502225 state 0x2 ref 0 00000100:00100000:3.0:1789760757.625623:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760757.627381:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495855872 00000100:00100000:1.0:1789760757.627397:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495855872 00000100:00100000:1.0:1789760757.627475:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741765807352 time 741849502225 state 0x0 ref 1 00000100:00100000:1.0:1789760757.627494:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741849762641 time 741849502225 state 0x1 ref 1 00000100:00100000:1.0:1789760757.627532:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741849762641 time 741948249289 state 0x0 ref 1 00000100:00100000:1.0:1789760757.627546:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495855872:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760757.627600:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24780800, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc007 00010000:00010000:1.0:1789760757.627610:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d3a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.627627:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760757.627634:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.627641:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.627655:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760757.683957:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973367 is committed 00002000:00010000:1.0:1789760757.684164:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24780800 -> 24784896 00002000:00010000:1.0:1789760757.684170:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760756 -> 1789760757 00002000:00010000:1.0:1789760757.684173:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760756 -> 1789760757 00010000:00010000:1.0:1789760757.684191:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.684213:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d3a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.684910:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.684928:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760757.684934:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.684946:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d3a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.684978:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d3a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.684997:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d3a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.685141:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495855872:12345-192.168.203.9@tcp:4:dd.0 Request processed in 57590us (59523us total) trans 4294973367 rc 0/0 00000100:00100000:1.0:1789760757.685155:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6079 00000100:00100000:1.0:1789760757.685169:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741849762641 time 741948249289 state 0x2 ref 0 00000100:00100000:2.0:1789760757.693463:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760757.693707:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495856000 00000100:00100000:1.0:1789760757.693720:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495856000 00000100:00100000:1.0:1789760757.693796:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741849762641 time 741948249289 state 0x0 ref 1 00000100:00100000:1.0:1789760757.693815:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741948509705 time 741948249289 state 0x1 ref 1 00000100:00100000:1.0:1789760757.693868:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741948509705 time 742014583894 state 0x0 ref 1 00000100:00100000:1.0:1789760757.693882:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495856000:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760757.693960:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24784896, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc007 00010000:00010000:1.0:1789760757.693971:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d41 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.693999:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760757.694004:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.694012:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.694029:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760757.740751:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973368 is committed 00002000:00010000:1.0:1789760757.740829:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24784896 -> 24788992 00010000:00010000:1.0:1789760757.740848:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.740862:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d41 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.740877:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.740885:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760757.740890:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.740901:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d41 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.740943:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d41 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.740950:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d41 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.741078:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495856000:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47190us (47620us total) trans 4294973368 rc 0/0 00000100:00100000:1.0:1789760757.741092:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6080 00000100:00100000:1.0:1789760757.741104:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741948509705 time 742014583894 state 0x2 ref 0 00000100:00100000:2.0:1789760757.746189:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760757.746365:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495856128 00000100:00100000:1.0:1789760757.746379:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495856128 00000100:00100000:1.0:1789760757.746452:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 741948509705 time 742014583894 state 0x0 ref 1 00000100:00100000:1.0:1789760757.746472:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742014844310 time 742014583894 state 0x1 ref 1 00000100:00100000:1.0:1789760757.746564:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742014844310 time 742067278786 state 0x0 ref 1 00000100:00100000:1.0:1789760757.746579:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495856128:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760757.746641:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24788992, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc007 00010000:00010000:1.0:1789760757.746651:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d48 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.746669:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760757.746674:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.746682:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.746698:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760757.813770:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24788992 -> 24793088 00010000:00010000:2.0:1789760757.813792:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.813807:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d48 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.813816:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.813830:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760757.813836:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.813847:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d48 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.813867:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d48 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.813876:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546d48 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.813985:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495856128:12345-192.168.203.9@tcp:4:dd.0 Request processed in 67404us (67809us total) trans 4294973369 rc 0/0 00000100:00100000:2.0:1789760757.813996:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6081 00000100:00100000:2.0:1789760757.814005:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742014844310 time 742067278786 state 0x2 ref 0 00000001:00080000:1.0:1789760757.814744:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973369 is committed 00000100:00100000:1.0:1789760757.818714:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760757.818794:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495856256 00000100:00100000:0.0:1789760757.818804:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495856256 00000100:00100000:0.0:1789760757.818859:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742014844310 time 742067278786 state 0x0 ref 1 00000100:00100000:0.0:1789760757.818874:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742067539202 time 742067278786 state 0x1 ref 1 00000100:00100000:0.0:1789760757.818905:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742067539202 time 742139621497 state 0x0 ref 1 00000100:00100000:0.0:1789760757.818926:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495856256:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760757.818973:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24793088, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc007 00010000:00010000:0.0:1789760757.818983:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546d4f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.818998:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:0.0:1789760757.819003:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.819009:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546d4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.819023:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546d4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760757.851617:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24793088 -> 24797184 00010000:00010000:3.0:1789760757.851647:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546d4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.851661:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546d4f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.851671:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546d4f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.851679:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ae00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760757.851684:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546d4f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.851692:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546d4f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.851711:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546d4f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.851724:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546d4f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.851818:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495856256:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32898us (33108us total) trans 4294973370 rc 0/0 00000100:00100000:3.0:1789760757.851829:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6082 00000100:00100000:3.0:1789760757.851840:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742067539202 time 742139621497 state 0x2 ref 0 00000001:00080000:1.0:1789760757.852321:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973370 is committed 00000100:00100000:3.0:1789760757.861787:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760757.862003:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495856384 00000100:00100000:1.0:1789760757.862013:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495856384 00000100:00100000:1.0:1789760757.862066:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742067539202 time 742139621497 state 0x0 ref 1 00000100:00100000:1.0:1789760757.862082:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742139881913 time 742139621497 state 0x1 ref 1 00000100:00100000:1.0:1789760757.862113:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742139881913 time 742182829939 state 0x0 ref 1 00000100:00100000:1.0:1789760757.862128:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495856384:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760757.862179:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24797184, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc007 00010000:00010000:1.0:1789760757.862188:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546d56 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.862204:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760757.862209:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.862215:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546d56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.862229:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546d56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.890203:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973371 is committed 00002000:00010000:2.0:1789760757.890406:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24797184 -> 24801280 00010000:00010000:2.0:1789760757.890422:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546d56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.890436:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546d56 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.890450:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546d56 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.890457:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760757.890462:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546d56 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.890472:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546d56 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.890490:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546d56 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.890498:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546d56 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.890625:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495856384:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28493us (28841us total) trans 4294973371 rc 0/0 00000100:00100000:2.0:1789760757.890636:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6083 00000100:00100000:2.0:1789760757.890646:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742139881913 time 742182829939 state 0x2 ref 0 00000100:00100000:2.0:1789760757.897602:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760757.897701:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495856512 00000100:00100000:1.0:1789760757.897711:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495856512 00000100:00100000:1.0:1789760757.897767:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742139881913 time 742182829939 state 0x0 ref 1 00000100:00100000:1.0:1789760757.897782:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742183090355 time 742182829939 state 0x1 ref 1 00000100:00100000:1.0:1789760757.897811:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742183090355 time 742218527934 state 0x0 ref 1 00000100:00100000:1.0:1789760757.897824:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495856512:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760757.897870:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24801280, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc007 00010000:00010000:1.0:1789760757.897879:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546d5d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.897906:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760757.897911:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.897927:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546d5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.897943:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546d5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.928498:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973372 is committed 00002000:00010000:1.0:1789760757.928549:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24801280 -> 24805376 00010000:00010000:1.0:1789760757.928564:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546d5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.928577:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546d5d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.928585:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546d5d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.928592:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760757.928596:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546d5d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.928604:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546d5d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.928623:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546d5d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.928631:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546d5d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.928725:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495856512:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30898us (31127us total) trans 4294973372 rc 0/0 00000100:00100000:1.0:1789760757.928735:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6084 00000100:00100000:1.0:1789760757.928744:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742183090355 time 742218527934 state 0x2 ref 0 00000100:00100000:1.0:1789760757.937165:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760757.937384:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495856640 00000100:00100000:3.0:1789760757.937393:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495856640 00000100:00100000:3.0:1789760757.937443:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742183090355 time 742218527934 state 0x0 ref 1 00000100:00100000:3.0:1789760757.937458:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742218788350 time 742218527934 state 0x1 ref 1 00000100:00100000:3.0:1789760757.937657:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742218788350 time 742258372083 state 0x0 ref 1 00000100:00100000:3.0:1789760757.937670:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495856640:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760757.937715:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24805376, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc007 00010000:00010000:3.0:1789760757.937724:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546d64 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.937740:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760757.937745:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.937753:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546d64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.937766:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546d64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.973521:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973373 is committed 00002000:00010000:1.0:1789760757.973667:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24805376 -> 24809472 00010000:00010000:1.0:1789760757.973682:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546d64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.973695:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546d64 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.973705:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546d64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.973713:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760757.973718:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546d64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.973727:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546d64 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.973751:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546d64 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.973758:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546d64 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.973938:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495856640:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36265us (36779us total) trans 4294973373 rc 0/0 00000100:00100000:1.0:1789760757.973948:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6085 00000100:00100000:1.0:1789760757.973958:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742218788350 time 742258372083 state 0x2 ref 0 00000100:00100000:2.0:1789760757.983774:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760757.984101:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495856768 00000100:00100000:1.0:1789760757.984111:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495856768 00000100:00100000:1.0:1789760757.984165:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742218788350 time 742258372083 state 0x0 ref 1 00000100:00100000:1.0:1789760757.984180:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742258632499 time 742258372083 state 0x1 ref 1 00000100:00100000:3.0:1789760757.984328:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742258632499 time 742305041970 state 0x0 ref 1 00000100:00100000:3.0:1789760757.984342:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495856768:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760757.984387:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24809472, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc007 00010000:00010000:3.0:1789760757.984401:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546d6b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.984417:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760757.984422:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.984428:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546d6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.984443:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546d6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760758.039158:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973374 is committed 00002000:00010000:1.0:1789760758.039309:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24809472 -> 24813568 00010000:00010000:1.0:1789760758.039324:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546d6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.039337:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546d6b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.039346:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546d6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.039359:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760758.039364:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546d6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.039374:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546d6b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.039393:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546d6b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.039402:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546d6b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.039483:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495856768:12345-192.168.203.9@tcp:4:dd.0 Request processed in 55139us (55713us total) trans 4294973374 rc 0/0 00000100:00100000:1.0:1789760758.039492:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6086 00000100:00100000:1.0:1789760758.039502:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742258632499 time 742305041970 state 0x2 ref 0 00000100:00100000:2.0:1789760758.044894:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760758.045020:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495856896 00000100:00100000:1.0:1789760758.045030:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495856896 00000100:00100000:1.0:1789760758.045080:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742258632499 time 742305041970 state 0x0 ref 1 00000100:00100000:1.0:1789760758.045094:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742305302386 time 742305041970 state 0x1 ref 1 00000100:00100000:1.0:1789760758.045121:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742305302386 time 742365838233 state 0x0 ref 1 00000100:00100000:1.0:1789760758.045136:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495856896:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760758.045183:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24813568, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc007 00010000:00010000:1.0:1789760758.045191:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546d72 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.045206:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760758.045210:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.045216:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546d72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.045229:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546d72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.047347:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033cfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3318:1876699520310272:0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.047415:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760758.047467:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff92903f925c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3318:1876699520310784:0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.047494:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760758.047519:0:3318:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1789760758.047525:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033ced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:59c4e264-9b75-482b-8813-3d1dc124f384:3318:1876699520310144:0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.047555:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760758.047577:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff92903f924380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3318:1876699520310656:0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.047596:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760758.047620:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033cd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520310400:0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.049188:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760758.049458:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff92903f926d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520310528:0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.049498:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760758.049536:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520310272 00000100:00100000:3.0:1789760758.049542:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520310272 00000100:00100000:3.0:1789760758.049576:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 497 00000100:00100000:3.0:1789760758.049583:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3318:x1876699520310272:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.049631:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3318:x1876699520310272:12345-0@lo:400:kworker.0 Request processed in 49us (2217us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760758.050145:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 497 00000100:00100000:3.0:1789760758.050164:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520310656 00000100:00100000:3.0:1789760758.050167:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520310656 00000100:00100000:3.0:1789760758.050190:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520310400 00000100:00100000:3.0:1789760758.050192:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520310400 00000100:00100000:3.0:1789760758.050203:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 498 00000100:00100000:3.0:1789760758.050208:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1876699520310656:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.050241:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1876699520310656:12345-0@lo:400:kworker.0 Request processed in 32us (2644us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760758.050246:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 498 00000100:00100000:3.0:1789760758.050256:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 499 00000100:00100000:3.0:1789760758.050261:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520310400:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.050297:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520310400:12345-0@lo:400:kworker.0 Request processed in 36us (1113us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760758.050303:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 499 00000100:00100000:3.0:1789760758.050318:0:10490:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520310784 00000100:00100000:3.0:1789760758.050321:0:10490:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520310784 00000100:00100000:3.0:1789760758.050342:0:10490:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520310528 00000100:00100000:3.0:1789760758.050344:0:10490:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520310528 00000100:00100000:3.0:1789760758.050361:0:10490:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 230 00000100:00100000:3.0:1789760758.050367:0:10490:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3318:x1876699520310784:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.050393:0:10490:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3318:x1876699520310784:12345-0@lo:400:kworker.0 Request processed in 27us (2899us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760758.050398:0:10490:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 230 00000100:00100000:3.0:1789760758.050406:0:10490:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 231 00000100:00100000:3.0:1789760758.050411:0:10490:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520310528:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.050434:0:10490:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520310528:12345-0@lo:400:kworker.0 Request processed in 24us (936us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760758.050440:0:10490:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 231 00000100:00100000:3.0:1789760758.050454:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520310144 00000100:00100000:3.0:1789760758.050456:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520310144 00000100:00100000:3.0:1789760758.050475:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 407 00000100:00100000:3.0:1789760758.050481:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929012311050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:59c4e264-9b75-482b-8813-3d1dc124f384+19:3318:x1876699520310144:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.050504:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929012311050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:59c4e264-9b75-482b-8813-3d1dc124f384+19:3318:x1876699520310144:12345-0@lo:400:kworker.0 Request processed in 23us (2950us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760758.050510:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 407 00000100:00080000:3.0:1789760758.052451:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789760758.052458:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789760758.052461:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760758.052512:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff92903f926d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520310528:0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.052556:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033ced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:59c4e264-9b75-482b-8813-3d1dc124f384:3318:1876699520310144:0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.052569:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff92903f924380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3318:1876699520310656:0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.052580:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033cfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3318:1876699520310272:0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.052589:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff92903f925c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3318:1876699520310784:0@lo:400:kworker.0 00000100:00100000:3.0:1789760758.052623:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033cd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520310400:0@lo:400:kworker.0 00000001:00080000:3.0:1789760758.076664:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973375 is committed 00002000:00010000:2.0:1789760758.076875:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24813568 -> 24817664 00010000:00010000:2.0:1789760758.076890:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546d72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.076973:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546d72 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.076982:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546d72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.076989:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760758.076995:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546d72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.077003:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546d72 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.077023:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546d72 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.077029:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546d72 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.077123:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495856896:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31983us (32231us total) trans 4294973375 rc 0/0 00000100:00100000:2.0:1789760758.077133:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6087 00000100:00100000:2.0:1789760758.077142:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742305302386 time 742365838233 state 0x2 ref 0 00000100:00100000:2.0:1789760758.082151:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760758.082269:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495857024 00000100:00100000:1.0:1789760758.082488:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495857024 00000100:00100000:1.0:1789760758.082543:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742305302386 time 742365838233 state 0x0 ref 1 00000100:00100000:1.0:1789760758.082557:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742366098649 time 742365838233 state 0x1 ref 1 00000100:00100000:1.0:1789760758.082617:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742366098649 time 742403332646 state 0x0 ref 1 00000100:00100000:1.0:1789760758.082630:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495857024:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760758.082688:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24817664, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc007 00010000:00010000:1.0:1789760758.082698:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546d79 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.082720:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760758.082725:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.082732:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546d79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.082745:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546d79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760758.114536:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973376 is committed 00002000:00010000:3.0:1789760758.114623:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24817664 -> 24821760 00010000:00010000:3.0:1789760758.114729:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546d79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.114743:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546d79 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.114753:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546d79 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.114760:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760758.114765:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546d79 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.114774:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546d79 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.114806:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546d79 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.114814:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546d79 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.115377:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495857024:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32260us (32748us total) trans 4294973376 rc 0/0 00000100:00100000:3.0:1789760758.115388:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6088 00000100:00100000:3.0:1789760758.115398:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742366098649 time 742403332646 state 0x2 ref 0 00000100:00100000:3.0:1789760758.119005:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760758.119204:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495857152 00000100:00100000:1.0:1789760758.119214:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495857152 00000100:00100000:1.0:1789760758.119269:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742366098649 time 742403332646 state 0x0 ref 1 00000100:00100000:1.0:1789760758.119325:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742403593062 time 742403332646 state 0x1 ref 1 00000100:00100000:1.0:1789760758.119357:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742403593062 time 742440073716 state 0x0 ref 1 00000100:00100000:1.0:1789760758.119370:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495857152:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760758.119418:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24821760, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc007 00010000:00010000:1.0:1789760758.119427:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546d80 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.119443:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760758.119448:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.119454:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546d80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.119474:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546d80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760758.148326:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24821760 -> 24825856 00010000:00010000:3.0:1789760758.148346:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546d80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.148358:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546d80 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.148366:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546d80 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.148373:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760758.148377:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546d80 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.148390:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546d80 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.148409:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546d80 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.148427:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546d80 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.148516:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495857152:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29144us (29515us total) trans 4294973377 rc 0/0 00000100:00100000:3.0:1789760758.148526:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6089 00000100:00100000:3.0:1789760758.148535:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742403593062 time 742440073716 state 0x2 ref 0 00000001:00080000:0.0:1789760758.148900:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973377 is committed 00000100:00100000:2.0:1789760758.155218:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760758.155379:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495857280 00000100:00100000:1.0:1789760758.155389:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495857280 00000100:00100000:1.0:1789760758.155450:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742403593062 time 742440073716 state 0x0 ref 1 00000100:00100000:1.0:1789760758.155466:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742440334132 time 742440073716 state 0x1 ref 1 00000100:00100000:1.0:1789760758.155531:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742440334132 time 742476248002 state 0x0 ref 1 00000100:00100000:1.0:1789760758.155544:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495857280:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760758.155602:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24825856, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc007 00010000:00010000:1.0:1789760758.155611:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546d87 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.155627:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760758.155631:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.155637:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546d87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.155650:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546d87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760758.188612:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24825856 -> 24829952 00010000:00010000:2.0:1789760758.188632:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546d87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.188645:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546d87 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.188654:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546d87 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.188663:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760758.188668:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546d87 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.188676:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546d87 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.188696:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546d87 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.188703:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354546d87 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.188795:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495857280:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33249us (33581us total) trans 4294973378 rc 0/0 00000100:00100000:2.0:1789760758.188807:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6090 00000100:00100000:2.0:1789760758.188816:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742440334132 time 742476248002 state 0x2 ref 0 00000001:00080000:2.0:1789760758.190361:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973378 is committed 00000100:00100000:2.0:1789760758.197719:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760758.197789:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495857408 00000100:00100000:3.0:1789760758.197800:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495857408 00000100:00100000:3.0:1789760758.197856:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742440334132 time 742476248002 state 0x0 ref 1 00000100:00100000:3.0:1789760758.197872:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742476508418 time 742476248002 state 0x1 ref 1 00000100:00100000:3.0:1789760758.197906:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742476508418 time 742518622419 state 0x0 ref 1 00000100:00100000:3.0:1789760758.197928:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495857408:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760758.197977:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24829952, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc007 00010000:00010000:3.0:1789760758.197986:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546d8e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.198003:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760758.198011:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.198017:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546d8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.198032:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546d8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760758.225680:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973379 is committed 00002000:00010000:1.0:1789760758.225782:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24829952 -> 24834048 00010000:00010000:1.0:1789760758.225799:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546d8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.225812:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546d8e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.225821:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546d8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.225829:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760758.225834:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546d8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.225844:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546d8e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.225864:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546d8e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.225872:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546d8e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.226038:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495857408:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28115us (28323us total) trans 4294973379 rc 0/0 00000100:00100000:1.0:1789760758.226050:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6091 00000100:00100000:1.0:1789760758.226061:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742476508418 time 742518622419 state 0x2 ref 0 00000100:00100000:1.0:1789760758.230882:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760758.231029:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495857536 00000100:00100000:3.0:1789760758.231039:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495857536 00000100:00100000:3.0:1789760758.231095:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742476508418 time 742518622419 state 0x0 ref 1 00000100:00100000:3.0:1789760758.231111:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742518882835 time 742518622419 state 0x1 ref 1 00000100:00100000:3.0:1789760758.231192:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742518882835 time 742551908188 state 0x0 ref 1 00000100:00100000:3.0:1789760758.231204:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495857536:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760758.231253:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24834048, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc007 00010000:00010000:3.0:1789760758.231262:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546d95 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.231323:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760758.231328:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.231335:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546d95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.231355:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546d95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760758.273270:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973380 is committed 00002000:00010000:2.0:1789760758.273363:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24834048 -> 24838144 00010000:00010000:2.0:1789760758.273378:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546d95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.273390:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546d95 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.273398:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546d95 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.273406:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760758.273411:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546d95 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.273419:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546d95 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.273446:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546d95 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.273461:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546d95 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.273551:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495857536:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42344us (42674us total) trans 4294973380 rc 0/0 00000100:00100000:2.0:1789760758.273561:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6092 00000100:00100000:2.0:1789760758.273572:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742518882835 time 742551908188 state 0x2 ref 0 00000100:00100000:2.0:1789760758.277943:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760758.278040:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495857664 00000100:00100000:1.0:1789760758.278050:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495857664 00000100:00100000:1.0:1789760758.278107:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742518882835 time 742551908188 state 0x0 ref 1 00000100:00100000:1.0:1789760758.278122:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742552168604 time 742551908188 state 0x1 ref 1 00000100:00100000:1.0:1789760758.278180:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742552168604 time 742598896980 state 0x0 ref 1 00000100:00100000:1.0:1789760758.278192:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495857664:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760758.278239:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24838144, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc007 00010000:00010000:1.0:1789760758.278248:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546d9c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.278262:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760758.278265:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.278270:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546d9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.278316:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546d9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760758.310572:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973381 is committed 00002000:00010000:2.0:1789760758.310689:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24838144 -> 24842240 00010000:00010000:2.0:1789760758.310704:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546d9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.310714:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546d9c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.310720:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546d9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.310724:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760758.310728:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546d9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.310734:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546d9c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.310753:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546d9c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.310758:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546d9c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.310861:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495857664:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32668us (32926us total) trans 4294973381 rc 0/0 00000100:00100000:2.0:1789760758.310868:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6093 00000100:00100000:2.0:1789760758.310875:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742552168604 time 742598896980 state 0x2 ref 0 00000100:00100000:1.0:1789760758.314791:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760758.314893:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495857792 00000100:00100000:0.0:1789760758.314903:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495857792 00000100:00100000:0.0:1789760758.314964:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742552168604 time 742598896980 state 0x0 ref 1 00000100:00100000:0.0:1789760758.314978:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742599157396 time 742598896980 state 0x1 ref 1 00000100:00100000:2.0:1789760758.315100:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742599157396 time 742635814246 state 0x0 ref 1 00000100:00100000:2.0:1789760758.315115:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495857792:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760758.316109:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24842240, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc007 00010000:00010000:3.0:1789760758.316122:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546da3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.316146:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760758.316150:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.316158:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546da3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.316172:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546da3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760758.351964:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973382 is committed 00002000:00010000:3.0:1789760758.352088:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24842240 -> 24846336 00010000:00010000:3.0:1789760758.352103:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546da3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.352115:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546da3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.352124:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546da3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.352137:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760758.352142:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546da3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.352152:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546da3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.352172:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546da3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.352179:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546da3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.352349:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495857792:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37228us (37559us total) trans 4294973382 rc 0/0 00000100:00100000:3.0:1789760758.352360:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6094 00000100:00100000:3.0:1789760758.352369:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742599157396 time 742635814246 state 0x2 ref 0 00000100:00100000:1.0:1789760758.356258:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760758.356546:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495857920 00000100:00100000:2.0:1789760758.356556:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495857920 00000100:00100000:2.0:1789760758.356610:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742599157396 time 742635814246 state 0x0 ref 1 00000100:00100000:2.0:1789760758.356626:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742636074662 time 742635814246 state 0x1 ref 1 00000100:00100000:2.0:1789760758.356660:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742636074662 time 742677376858 state 0x0 ref 1 00000100:00100000:2.0:1789760758.356672:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495857920:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760758.356731:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24846336, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc007 00010000:00010000:2.0:1789760758.356739:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546daa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.356754:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:2.0:1789760758.356759:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.356765:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546daa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.356779:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546daa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760758.380820:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24846336 -> 24850432 00010000:00010000:2.0:1789760758.380838:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546daa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.380852:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546daa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.380861:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546daa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.380868:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760758.380872:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546daa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.380882:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546daa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.380913:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546daa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.380929:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546daa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.381023:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495857920:12345-192.168.203.9@tcp:4:dd.0 Request processed in 24348us (24770us total) trans 4294973383 rc 0/0 00000100:00100000:2.0:1789760758.381034:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6095 00000100:00100000:2.0:1789760758.381044:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742636074662 time 742677376858 state 0x2 ref 0 00000001:00080000:2.0:1789760758.381087:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973383 is committed 00000100:00100000:2.0:1789760758.386023:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760758.386178:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495858048 00000100:00100000:1.0:1789760758.386189:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495858048 00000100:00100000:1.0:1789760758.386256:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742636074662 time 742677376858 state 0x0 ref 1 00000100:00100000:1.0:1789760758.386272:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742677637274 time 742677376858 state 0x1 ref 1 00000100:00100000:1.0:1789760758.386331:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742677637274 time 742707048058 state 0x0 ref 1 00000100:00100000:1.0:1789760758.386343:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495858048:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760758.386388:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24850432, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc007 00010000:00010000:1.0:1789760758.386398:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546db1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.386414:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760758.386418:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.386425:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546db1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.386438:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546db1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760758.415766:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973384 is committed 00002000:00010000:1.0:1789760758.415846:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24850432 -> 24854528 00010000:00010000:1.0:1789760758.415860:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546db1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.415872:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546db1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.415881:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546db1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.415889:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760758.415962:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546db1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.415972:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546db1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.416059:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546db1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.416067:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546db1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.416217:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495858048:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29870us (30207us total) trans 4294973384 rc 0/0 00000100:00100000:1.0:1789760758.416228:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6096 00000100:00100000:1.0:1789760758.416441:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742677637274 time 742707048058 state 0x2 ref 0 00000100:00100000:1.0:1789760758.419482:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760758.419565:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495858176 00000100:00100000:3.0:1789760758.419574:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495858176 00000100:00100000:3.0:1789760758.419622:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742677637274 time 742707048058 state 0x0 ref 1 00000100:00100000:3.0:1789760758.419640:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742707308474 time 742707048058 state 0x1 ref 1 00000100:00100000:3.0:1789760758.419688:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742707308474 time 742740405339 state 0x0 ref 1 00000100:00100000:3.0:1789760758.419702:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495858176:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760758.419743:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24854528, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc007 00010000:00010000:3.0:1789760758.419751:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546db8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.419766:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760758.419770:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.419776:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546db8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.419789:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546db8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760758.452178:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973385 is committed 00002000:00010000:1.0:1789760758.452353:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24854528 -> 24858624 00010000:00010000:1.0:1789760758.452369:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546db8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.452382:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546db8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.452396:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546db8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.452403:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760758.452408:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546db8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.452416:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546db8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.452434:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546db8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.452442:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546db8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.452545:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495858176:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32838us (33064us total) trans 4294973385 rc 0/0 00000100:00100000:1.0:1789760758.452556:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6097 00000100:00100000:1.0:1789760758.452566:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742707308474 time 742740405339 state 0x2 ref 0 00000100:00100000:2.0:1789760758.456406:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760758.456530:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495858304 00000100:00100000:0.0:1789760758.456540:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495858304 00000100:00100000:0.0:1789760758.456598:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742707308474 time 742740405339 state 0x0 ref 1 00000100:00100000:0.0:1789760758.456613:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742740665755 time 742740405339 state 0x1 ref 1 00000100:00100000:0.0:1789760758.456640:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742740665755 time 742777356887 state 0x0 ref 1 00000100:00100000:0.0:1789760758.456652:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495858304:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760758.456699:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24858624, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc007 00010000:00010000:0.0:1789760758.456708:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546dbf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.456723:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:0.0:1789760758.456727:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760758.456734:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546dbf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.456746:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546dbf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760758.495008:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973386 is committed 00002000:00010000:1.0:1789760758.495052:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24858624 -> 24862720 00010000:00010000:1.0:1789760758.495067:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546dbf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.495086:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546dbf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.495095:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546dbf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.495102:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94aa00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760758.495106:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546dbf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.495115:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546dbf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.495256:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546dbf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.495267:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354546dbf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.495567:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495858304:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38911us (39164us total) trans 4294973386 rc 0/0 00000100:00100000:2.0:1789760758.495578:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6098 00000100:00100000:2.0:1789760758.495588:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742740665755 time 742777356887 state 0x2 ref 0 00000100:00100000:3.0:1789760758.502894:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760758.503153:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495858432 00000100:00100000:1.0:1789760758.503169:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495858432 00000100:00100000:1.0:1789760758.503230:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742740665755 time 742777356887 state 0x0 ref 1 00000100:00100000:1.0:1789760758.503245:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742777617303 time 742777356887 state 0x1 ref 1 00000100:00100000:1.0:1789760758.503308:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742777617303 time 742823996357 state 0x0 ref 1 00000100:00100000:1.0:1789760758.503322:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495858432:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760758.503369:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24862720, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc007 00010000:00010000:1.0:1789760758.503378:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546dc6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.503394:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760758.503398:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.503405:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546dc6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.503418:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546dc6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760758.538920:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24862720 -> 24866816 00010000:00010000:1.0:1789760758.538941:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546dc6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.538954:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546dc6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.538963:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546dc6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.538971:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760758.538975:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546dc6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.538985:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546dc6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.539004:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546dc6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.539012:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354546dc6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.539238:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495858432:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35913us (36347us total) trans 4294973387 rc 0/0 00000100:00100000:1.0:1789760758.539249:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6099 00000100:00100000:1.0:1789760758.539261:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742777617303 time 742823996357 state 0x2 ref 0 00000001:00080000:0.0:1789760758.540000:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973387 is committed 00000100:00100000:1.0:1789760758.547672:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760758.547833:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495858560 00000100:00100000:3.0:1789760758.547844:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495858560 00000100:00100000:3.0:1789760758.547907:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742777617303 time 742823996357 state 0x0 ref 1 00000100:00100000:3.0:1789760758.547932:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742824256773 time 742823996357 state 0x1 ref 1 00000100:00100000:3.0:1789760758.547961:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742824256773 time 742868677247 state 0x0 ref 1 00000100:00100000:3.0:1789760758.547973:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034ce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495858560:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760758.548016:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24866816, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc007 00010000:00010000:3.0:1789760758.548026:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546dcd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.548267:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760758.548273:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.548517:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546dcd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.548532:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546dcd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.559058:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f01835c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520310912:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760758.559726:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760758.560250:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520310912 00000100:00100000:0.0:1789760758.560260:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520310912 00000100:00100000:0.0:1789760758.560323:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 206 00000100:00100000:0.0:1789760758.560333:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fba2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520310912:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789760758.560366:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760758.560417:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fba2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520310912:12345-0@lo:13:osp-pre-1-0.0 Request processed in 86us (692us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760758.560424:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 206 00000100:00100000:3.0:1789760758.560522:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f01835c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520310912:0@lo:13:osp-pre-1-0.0 00002000:00010000:3.0:1789760758.599125:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24866816 -> 24870912 00010000:00010000:3.0:1789760758.599147:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546dcd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.599162:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546dcd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.599171:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546dcd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.599178:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760758.599182:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546dcd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.599192:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546dcd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.599212:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546dcd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.599219:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546dcd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.599428:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034ce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495858560:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51452us (51759us total) trans 4294973388 rc 0/0 00000100:00100000:3.0:1789760758.599439:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6100 00000100:00100000:3.0:1789760758.599449:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742824256773 time 742868677247 state 0x2 ref 0 00000001:00080000:1.0:1789760758.600552:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973388 is committed 00000100:00100000:3.0:1789760758.606719:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760758.606830:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495858688 00000100:00100000:2.0:1789760758.606840:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495858688 00000100:00100000:2.0:1789760758.606892:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742824256773 time 742868677247 state 0x0 ref 1 00000100:00100000:2.0:1789760758.606926:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742868937663 time 742868677247 state 0x1 ref 1 00000100:00100000:2.0:1789760758.606958:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742868937663 time 742927675012 state 0x0 ref 1 00000100:00100000:2.0:1789760758.606970:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495858688:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760758.607023:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24870912, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc007 00010000:00010000:2.0:1789760758.607032:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546dd4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.607048:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:2.0:1789760758.607053:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.607059:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546dd4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.607072:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546dd4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.623224:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760758.650806:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973389 is committed 00002000:00010000:3.0:1789760758.651080:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24870912 -> 24875008 00010000:00010000:3.0:1789760758.651095:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546dd4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.651107:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546dd4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.651115:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546dd4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.651126:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760758.651130:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546dd4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.651138:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546dd4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.651156:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546dd4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.651162:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546dd4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.651323:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495858688:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44349us (44612us total) trans 4294973389 rc 0/0 00000100:00100000:3.0:1789760758.651333:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6101 00000100:00100000:3.0:1789760758.651342:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742868937663 time 742927675012 state 0x2 ref 0 00000100:00100000:3.0:1789760758.658170:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760758.658556:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495858816 00000100:00100000:1.0:1789760758.658566:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495858816 00000100:00100000:1.0:1789760758.658627:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742868937663 time 742927675012 state 0x0 ref 1 00000100:00100000:1.0:1789760758.658642:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742927935428 time 742927675012 state 0x1 ref 1 00000100:00100000:1.0:1789760758.658732:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742927935428 time 742979447560 state 0x0 ref 1 00000100:00100000:1.0:1789760758.658745:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495858816:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760758.658794:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24875008, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc007 00010000:00010000:1.0:1789760758.658804:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ddb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.658820:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760758.658824:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.658831:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ddb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.658845:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ddb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760758.703870:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973390 is committed 00002000:00010000:1.0:1789760758.703993:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24875008 -> 24879104 00002000:00010000:1.0:1789760758.703998:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760757 -> 1789760758 00002000:00010000:1.0:1789760758.704001:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760757 -> 1789760758 00010000:00010000:1.0:1789760758.704018:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ddb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.704031:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ddb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.704040:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ddb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.704047:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760758.704052:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ddb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.704066:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ddb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.704089:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ddb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.704098:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ddb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.704196:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495858816:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45444us (46025us total) trans 4294973390 rc 0/0 00000100:00100000:1.0:1789760758.704204:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6102 00000100:00100000:1.0:1789760758.704212:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742927935428 time 742979447560 state 0x2 ref 0 00000100:00100000:1.0:1789760758.708253:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760758.708442:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495858944 00000100:00100000:3.0:1789760758.708452:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495858944 00000100:00100000:3.0:1789760758.708507:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742927935428 time 742979447560 state 0x0 ref 1 00000100:00100000:3.0:1789760758.708522:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742979707976 time 742979447560 state 0x1 ref 1 00000100:00100000:3.0:1789760758.708624:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742979707976 time 743029339862 state 0x0 ref 1 00000100:00100000:3.0:1789760758.708638:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495858944:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760758.708690:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24879104, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc007 00010000:00010000:3.0:1789760758.708699:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546de2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.708715:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760758.708720:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.708726:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546de2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.708737:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546de2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760758.746459:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973391 is committed 00002000:00010000:3.0:1789760758.746514:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24879104 -> 24883200 00010000:00010000:3.0:1789760758.746531:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546de2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.746544:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546de2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.746554:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546de2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.746561:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760758.746577:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546de2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.746587:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546de2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.746610:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546de2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.746618:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546de2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.746707:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495858944:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38067us (38458us total) trans 4294973391 rc 0/0 00000100:00100000:3.0:1789760758.746717:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6103 00000100:00100000:3.0:1789760758.746726:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742979707976 time 743029339862 state 0x2 ref 0 00000100:00100000:3.0:1789760758.751831:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760758.752517:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495859072 00000100:00100000:2.0:1789760758.752529:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495859072 00000100:00100000:2.0:1789760758.752597:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 742979707976 time 743029339862 state 0x0 ref 1 00000100:00100000:2.0:1789760758.752611:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743029600278 time 743029339862 state 0x1 ref 1 00000100:00100000:2.0:1789760758.752642:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743029600278 time 743073359628 state 0x0 ref 1 00000100:00100000:2.0:1789760758.752654:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495859072:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760758.752697:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24883200, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc007 00010000:00010000:2.0:1789760758.752706:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546de9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.752723:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:2.0:1789760758.752727:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.752733:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546de9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.752745:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546de9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760758.800870:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24883200 -> 24887296 00010000:00010000:2.0:1789760758.800900:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546de9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.800929:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546de9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.800938:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546de9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.800946:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760758.800951:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546de9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.800959:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546de9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.801102:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546de9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.801109:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546de9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.801202:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495859072:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48546us (49376us total) trans 4294973392 rc 0/0 00000100:00100000:2.0:1789760758.801212:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6104 00000100:00100000:2.0:1789760758.801221:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743029600278 time 743073359628 state 0x2 ref 0 00000001:00080000:3.0:1789760758.802270:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973392 is committed 00000100:00100000:2.0:1789760758.805467:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760758.805561:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495859200 00000100:00100000:1.0:1789760758.805572:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495859200 00000100:00100000:1.0:1789760758.805626:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743029600278 time 743073359628 state 0x0 ref 1 00000100:00100000:1.0:1789760758.805641:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743073620044 time 743073359628 state 0x1 ref 1 00000100:00100000:1.0:1789760758.805671:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743073620044 time 743126387885 state 0x0 ref 1 00000100:00100000:1.0:1789760758.805683:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495859200:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760758.805733:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24887296, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc007 00010000:00010000:1.0:1789760758.805742:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546df0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.805760:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:1.0:1789760758.805765:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.805783:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546df0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.805798:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546df0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760758.833589:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24887296 -> 24891392 00010000:00010000:1.0:1789760758.833612:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546df0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.833625:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546df0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.833634:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546df0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.833641:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760758.833646:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546df0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.833655:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546df0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.833685:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546df0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.833692:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546df0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.833792:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495859200:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28106us (28329us total) trans 4294973393 rc 0/0 00000100:00100000:1.0:1789760758.833803:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6105 00000100:00100000:1.0:1789760758.833812:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743073620044 time 743126387885 state 0x2 ref 0 00000001:00080000:2.0:1789760758.833887:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973393 is committed 00000100:00100000:1.0:1789760758.839239:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760758.840475:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495859328 00000100:00100000:3.0:1789760758.840489:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495859328 00000100:00100000:3.0:1789760758.840632:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743073620044 time 743126387885 state 0x0 ref 1 00000100:00100000:3.0:1789760758.840648:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743126648301 time 743126387885 state 0x1 ref 1 00000100:00100000:3.0:1789760758.840749:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743126648301 time 743161463387 state 0x0 ref 1 00000100:00100000:3.0:1789760758.840766:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495859328:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760758.840823:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24891392, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc007 00010000:00010000:3.0:1789760758.840836:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546df7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.840853:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760758.840858:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.840864:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546df7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.840878:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546df7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760758.877064:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24891392 -> 24895488 00010000:00010000:3.0:1789760758.877083:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546df7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.877097:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546df7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.877107:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546df7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.877114:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760758.877119:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546df7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.877128:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546df7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.877149:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546df7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.877156:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546df7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.877259:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495859328:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36493us (38023us total) trans 4294973394 rc 0/0 00000100:00100000:3.0:1789760758.877270:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6106 00000100:00100000:3.0:1789760758.877318:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743126648301 time 743161463387 state 0x2 ref 0 00000001:00080000:2.0:1789760758.878384:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973394 is committed 00000100:00100000:3.0:1789760758.884621:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760758.884739:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495859456 00000100:00100000:2.0:1789760758.884749:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495859456 00000100:00100000:2.0:1789760758.884803:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743126648301 time 743161463387 state 0x0 ref 1 00000100:00100000:2.0:1789760758.884819:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743161723803 time 743161463387 state 0x1 ref 1 00000100:00100000:2.0:1789760758.884854:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743161723803 time 743205570583 state 0x0 ref 1 00000100:00100000:2.0:1789760758.884868:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495859456:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760758.885040:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24895488, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc007 00010000:00010000:2.0:1789760758.885049:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546dfe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.885066:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:2.0:1789760758.885070:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.885076:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546dfe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.885089:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546dfe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760758.937360:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24895488 -> 24899584 00010000:00010000:2.0:1789760758.937380:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546dfe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.937394:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546dfe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.937402:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546dfe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.937410:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760758.937414:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546dfe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.937424:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546dfe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.937444:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546dfe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.937452:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546dfe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.937544:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495859456:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52674us (52926us total) trans 4294973395 rc 0/0 00000100:00100000:2.0:1789760758.937555:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6107 00000100:00100000:2.0:1789760758.937563:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743161723803 time 743205570583 state 0x2 ref 0 00000001:00080000:0.0:1789760758.937644:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973395 is committed 00000100:00100000:0.0:1789760758.941204:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760758.941343:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495859584 00000100:00100000:1.0:1789760758.941353:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495859584 00000100:00100000:1.0:1789760758.941406:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743161723803 time 743205570583 state 0x0 ref 1 00000100:00100000:1.0:1789760758.941421:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743205830999 time 743205570583 state 0x1 ref 1 00000100:00100000:1.0:1789760758.941449:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743205830999 time 743262166329 state 0x0 ref 1 00000100:00100000:1.0:1789760758.941461:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495859584:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760758.941510:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24899584, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc007 00010000:00010000:1.0:1789760758.941519:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546e05 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.941541:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:1.0:1789760758.941545:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.941551:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546e05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.941564:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546e05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760758.973093:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24899584 -> 24903680 00010000:00010000:2.0:1789760758.973111:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546e05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.973123:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546e05 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.973131:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546e05 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.973137:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760758.973142:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546e05 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.973150:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546e05 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.973173:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546e05 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.973180:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546e05 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.973275:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495859584:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31811us (32075us total) trans 4294973396 rc 0/0 00000100:00100000:2.0:1789760758.973328:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6108 00000100:00100000:2.0:1789760758.973338:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743205830999 time 743262166329 state 0x2 ref 0 00000001:00080000:0.0:1789760758.974187:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973396 is committed 00000100:00100000:1.0:1789760758.977689:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760758.977778:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495859712 00000100:00100000:3.0:1789760758.977788:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495859712 00000100:00100000:3.0:1789760758.977841:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743205830999 time 743262166329 state 0x0 ref 1 00000100:00100000:3.0:1789760758.977857:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743262426745 time 743262166329 state 0x1 ref 1 00000100:00100000:3.0:1789760758.977885:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743262426745 time 743298601443 state 0x0 ref 1 00000100:00100000:3.0:1789760758.977904:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495859712:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760758.977961:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24903680, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc007 00010000:00010000:3.0:1789760758.977970:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546e0c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.977990:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760758.977994:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.978001:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546e0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.978014:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546e0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760759.009745:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24903680 -> 24907776 00010000:00010000:1.0:1789760759.009768:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546e0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.009781:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546e0c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.009790:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546e0c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.009797:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760759.009802:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546e0c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760759.009807:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973397 is committed 00010000:00010000:1.0:1789760759.009810:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546e0c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.009834:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546e0c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.009843:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546e0c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.010049:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495859712:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32142us (32365us total) trans 4294973397 rc 0/0 00000100:00100000:1.0:1789760759.010060:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6109 00000100:00100000:1.0:1789760759.010070:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743262426745 time 743298601443 state 0x2 ref 0 00000100:00100000:1.0:1789760759.029404:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760759.029506:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495859840 00000100:00100000:2.0:1789760759.029515:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495859840 00000100:00100000:2.0:1789760759.029591:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743262426745 time 743298601443 state 0x0 ref 1 00000100:00100000:2.0:1789760759.029606:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743298861859 time 743298601443 state 0x1 ref 1 00000100:00100000:2.0:1789760759.029627:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743298861859 time 743350344248 state 0x0 ref 1 00000100:00100000:2.0:1789760759.029638:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495859840:12345-192.168.203.9@tcp:4:dd.0 00000100:00080000:3.0:1789760759.029663:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789760759.029686:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24907776, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc007 00010000:00010000:2.0:1789760759.029695:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546e13 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.029711:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:2.0:1789760759.029716:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.029722:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546e13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.029742:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546e13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760759.069373:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973398 is committed 00002000:00010000:2.0:1789760759.069805:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24907776 -> 24911872 00010000:00010000:2.0:1789760759.069822:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546e13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.069834:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546e13 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.069844:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546e13 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.069851:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760759.069856:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546e13 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.069865:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546e13 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.069885:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546e13 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.069893:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546e13 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.070077:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495859840:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40435us (40677us total) trans 4294973398 rc 0/0 00000100:00100000:2.0:1789760759.070087:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6110 00000100:00100000:2.0:1789760759.070097:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743298861859 time 743350344248 state 0x2 ref 0 00000100:00100000:2.0:1789760759.073533:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760759.073700:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495859968 00000100:00100000:1.0:1789760759.073710:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495859968 00000100:00100000:1.0:1789760759.073764:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743298861859 time 743350344248 state 0x0 ref 1 00000100:00100000:1.0:1789760759.073779:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743350604664 time 743350344248 state 0x1 ref 1 00000100:00100000:1.0:1789760759.073840:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743350604664 time 743394556534 state 0x0 ref 1 00000100:00100000:1.0:1789760759.073853:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495859968:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760759.073920:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24911872, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc007 00010000:00010000:1.0:1789760759.073930:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546e1a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.073951:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:1.0:1789760759.073956:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.073962:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546e1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.073977:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546e1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760759.100269:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973399 is committed 00002000:00010000:1.0:1789760759.100350:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24911872 -> 24915968 00010000:00010000:1.0:1789760759.100364:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546e1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.100378:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546e1a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.100387:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546e1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.100394:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760759.100399:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546e1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.100409:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546e1a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.100433:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546e1a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.100441:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354546e1a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.100515:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495859968:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26660us (26987us total) trans 4294973399 rc 0/0 00000100:00100000:1.0:1789760759.100524:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6111 00000100:00100000:1.0:1789760759.100534:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743350604664 time 743394556534 state 0x2 ref 0 00000100:00100000:1.0:1789760759.105641:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760759.105839:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495860096 00000100:00100000:3.0:1789760759.105849:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495860096 00000100:00100000:3.0:1789760759.105906:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743350604664 time 743394556534 state 0x0 ref 1 00000100:00100000:3.0:1789760759.105929:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743394816950 time 743394556534 state 0x1 ref 1 00000100:00100000:3.0:1789760759.105959:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743394816950 time 743426675617 state 0x0 ref 1 00000100:00100000:3.0:1789760759.105972:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495860096:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760759.106016:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24915968, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc007 00010000:00010000:3.0:1789760759.106025:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546e21 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.106041:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760759.106045:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.106051:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546e21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.106071:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546e21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760759.127863:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973400 is committed 00002000:00010000:0.0:1789760759.128005:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24915968 -> 24920064 00010000:00010000:0.0:1789760759.128019:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546e21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.128033:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546e21 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.128042:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546e21 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.128049:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760759.128054:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546e21 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.128064:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546e21 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.128082:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546e21 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.128096:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546e21 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.128173:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495860096:12345-192.168.203.9@tcp:4:dd.0 Request processed in 22199us (22536us total) trans 4294973400 rc 0/0 00000100:00100000:0.0:1789760759.128182:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6112 00000100:00100000:0.0:1789760759.128192:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743394816950 time 743426675617 state 0x2 ref 0 00000100:00100000:1.0:1789760759.131462:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760759.131713:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495860224 00000100:00100000:2.0:1789760759.131722:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495860224 00000100:00100000:2.0:1789760759.131776:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743394816950 time 743426675617 state 0x0 ref 1 00000100:00100000:2.0:1789760759.131792:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743426936033 time 743426675617 state 0x1 ref 1 00000100:00100000:2.0:1789760759.131822:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743426936033 time 743452539461 state 0x0 ref 1 00000100:00100000:2.0:1789760759.131835:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495860224:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760759.131877:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24920064, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc007 00010000:00010000:2.0:1789760759.131886:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546e28 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.132056:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:2.0:1789760759.132060:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.132067:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546e28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.132081:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546e28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760759.159715:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973401 is committed 00002000:00010000:0.0:1789760759.159813:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24920064 -> 24924160 00010000:00010000:0.0:1789760759.159829:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546e28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.159844:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546e28 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.159854:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546e28 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.159863:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760759.159867:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546e28 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.159877:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546e28 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.159923:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546e28 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.159932:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354546e28 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.160030:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495860224:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28193us (28572us total) trans 4294973401 rc 0/0 00000100:00100000:0.0:1789760759.160041:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6113 00000100:00100000:0.0:1789760759.160051:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743426936033 time 743452539461 state 0x2 ref 0 00000100:00100000:1.0:1789760759.167561:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760759.167727:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495860352 00000100:00100000:2.0:1789760759.167737:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495860352 00000100:00100000:2.0:1789760759.167793:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743426936033 time 743452539461 state 0x0 ref 1 00000100:00100000:2.0:1789760759.167877:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743452799877 time 743452539461 state 0x1 ref 1 00000100:00100000:2.0:1789760759.167954:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743452799877 time 743488670563 state 0x0 ref 1 00000100:00100000:2.0:1789760759.167967:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495860352:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760759.168011:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24924160, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc007 00010000:00010000:2.0:1789760759.168020:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546e2f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.168036:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:2.0:1789760759.168040:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.168046:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546e2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.168060:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546e2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760759.196501:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973402 is committed 00002000:00010000:0.0:1789760759.196586:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24924160 -> 24928256 00010000:00010000:0.0:1789760759.196601:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546e2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.196614:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546e2f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.196623:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546e2f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.196631:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760759.196636:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546e2f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.196651:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546e2f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.196672:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546e2f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.196679:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546e2f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.196769:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495860352:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28799us (29211us total) trans 4294973402 rc 0/0 00000100:00100000:0.0:1789760759.196778:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6114 00000100:00100000:0.0:1789760759.196787:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743452799877 time 743488670563 state 0x2 ref 0 00000100:00100000:1.0:1789760759.200259:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760759.200393:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495860480 00000100:00100000:0.0:1789760759.200402:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495860480 00000100:00100000:0.0:1789760759.200452:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743452799877 time 743488670563 state 0x0 ref 1 00000100:00100000:0.0:1789760759.200477:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743488930979 time 743488670563 state 0x1 ref 1 00000100:00100000:0.0:1789760759.200495:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743488930979 time 743521213014 state 0x0 ref 1 00000100:00100000:0.0:1789760759.200507:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495860480:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760759.200553:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24928256, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc007 00010000:00010000:0.0:1789760759.200561:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546e36 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.200578:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760759.200582:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.200589:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546e36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.200602:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546e36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.230739:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973403 is committed 00002000:00010000:0.0:1789760759.230824:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24928256 -> 24932352 00010000:00010000:0.0:1789760759.230840:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546e36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.230852:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546e36 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.230862:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546e36 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.230870:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760759.230874:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546e36 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.230884:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546e36 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.230979:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546e36 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.230987:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546e36 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.231111:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495860480:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30600us (30855us total) trans 4294973403 rc 0/0 00000100:00100000:0.0:1789760759.231122:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6115 00000100:00100000:0.0:1789760759.231132:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743488930979 time 743521213014 state 0x2 ref 0 00000100:00100000:1.0:1789760759.236972:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760759.237062:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495860608 00000100:00100000:0.0:1789760759.237071:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495860608 00000100:00100000:0.0:1789760759.237127:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743488930979 time 743521213014 state 0x0 ref 1 00000100:00100000:0.0:1789760759.237143:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743521473430 time 743521213014 state 0x1 ref 1 00000100:00100000:0.0:1789760759.237209:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743521473430 time 743557925114 state 0x0 ref 1 00000100:00100000:0.0:1789760759.237224:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495860608:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760759.237271:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24932352, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc007 00010000:00010000:0.0:1789760759.237321:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546e3d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.237337:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760759.237342:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.237349:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546e3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.237370:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546e3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760759.276561:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973404 is committed 00002000:00010000:0.0:1789760759.276576:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24932352 -> 24936448 00010000:00010000:0.0:1789760759.276591:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546e3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.276604:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546e3d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.276617:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546e3d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.276634:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760759.276640:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546e3d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.276649:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546e3d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.276670:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546e3d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.276678:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546e3d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.276781:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495860608:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39556us (39813us total) trans 4294973404 rc 0/0 00000100:00100000:0.0:1789760759.276791:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6116 00000100:00100000:0.0:1789760759.276801:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743521473430 time 743557925114 state 0x2 ref 0 00000100:00100000:1.0:1789760759.285717:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760759.285821:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760759.286077:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760759.286103:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495860736 00000100:00100000:2.0:1789760759.286113:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495860736 00000100:00100000:2.0:1789760759.286165:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743521473430 time 743557925114 state 0x0 ref 1 00000100:00100000:2.0:1789760759.286181:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743558185530 time 743557925114 state 0x1 ref 1 00000100:00100000:2.0:1789760759.286258:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743558185530 time 743606973244 state 0x0 ref 1 00000100:00100000:2.0:1789760759.286271:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495860736:12345-192.168.203.9@tcp:4:dd.0 00000100:00100000:3.0:1789760759.286277:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495860992 00000100:00100000:3.0:1789760759.286295:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495860992 00002000:00010000:2.0:1789760759.286333:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24936448, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc007 00000100:00100000:3.0:1789760759.286334:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 500 00000100:00100000:3.0:1789760759.286341:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:17022965-1e4f-46fb-915c-61730096b8e2+8:2357:x1876699495860992:12345-192.168.203.9@tcp:400:kworker.0 00010000:00010000:2.0:1789760759.286343:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546e44 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.286358:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:2.0:1789760759.286362:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.286369:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546e44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.286381:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546e44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.286400:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:17022965-1e4f-46fb-915c-61730096b8e2+8:2357:x1876699495860992:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 60us (325us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760759.286408:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 500 00000100:00080000:3.0:1789760759.286429:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760759.287694:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495860864 00000100:00100000:0.0:1789760759.287706:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495860864 00000100:00100000:0.0:1789760759.288608:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 408 00000100:00100000:0.0:1789760759.288618:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929012312850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2357:x1876699495860864:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:0.0:1789760759.288824:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929012312850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2357:x1876699495860864:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 206us (3114us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760759.288832:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 408 00000100:00100000:1.0:1789760759.292644:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760759.292722:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495861120 00000100:00100000:3.0:1789760759.292731:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495861120 00000100:00100000:3.0:1789760759.292807:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 501 00000100:00100000:3.0:1789760759.292817:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2356:x1876699495861120:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:3.0:1789760759.292883:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2356:x1876699495861120:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 67us (248us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760759.292890:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 501 00000100:00080000:0.0:1789760759.292982:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:3.0:1789760759.333636:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973405 is committed 00002000:00010000:0.0:1789760759.333742:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24936448 -> 24940544 00010000:00010000:0.0:1789760759.333758:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546e44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.333771:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546e44 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.333781:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546e44 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.333789:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760759.333794:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546e44 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.333804:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546e44 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.333824:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546e44 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.333832:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354546e44 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.334030:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495860736:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47756us (48203us total) trans 4294973405 rc 0/0 00000100:00100000:0.0:1789760759.334040:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6117 00000100:00100000:0.0:1789760759.334050:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743558185530 time 743606973244 state 0x2 ref 0 00000100:00100000:1.0:1789760759.340072:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760759.340164:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495861248 00000100:00100000:0.0:1789760759.340174:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495861248 00000100:00100000:0.0:1789760759.340231:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743558185530 time 743606973244 state 0x0 ref 1 00000100:00100000:0.0:1789760759.340246:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743607233660 time 743606973244 state 0x1 ref 1 00000100:00100000:0.0:1789760759.340275:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743607233660 time 743660991824 state 0x0 ref 1 00000100:00100000:0.0:1789760759.340320:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04575500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495861248:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760759.340369:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24940544, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc007 00010000:00010000:0.0:1789760759.340379:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546e4b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.340395:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760759.340399:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.340406:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546e4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.340420:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546e4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.366661:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973406 is committed 00002000:00010000:2.0:1789760759.366868:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24940544 -> 24944640 00010000:00010000:2.0:1789760759.366894:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546e4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.366916:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546e4b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.366927:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546e4b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.366934:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760759.366940:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546e4b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.366948:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546e4b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.366967:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546e4b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.366976:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354546e4b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.367064:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04575500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495861248:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26742us (26998us total) trans 4294973406 rc 0/0 00000100:00100000:2.0:1789760759.367074:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6118 00000100:00100000:2.0:1789760759.367084:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743607233660 time 743660991824 state 0x2 ref 0 00000100:00100000:2.0:1789760759.370481:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760759.370696:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495861376 00000100:00100000:1.0:1789760759.370706:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495861376 00000100:00100000:1.0:1789760759.370760:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743607233660 time 743660991824 state 0x0 ref 1 00000100:00100000:1.0:1789760759.370775:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743661252240 time 743660991824 state 0x1 ref 1 00000100:00100000:1.0:1789760759.370838:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743661252240 time 743691554193 state 0x0 ref 1 00000100:00100000:1.0:1789760759.370852:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495861376:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760759.370919:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24944640, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc007 00010000:00010000:1.0:1789760759.370928:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354546e52 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.370943:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:1.0:1789760759.370947:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.370954:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354546e52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.370967:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354546e52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760759.395586:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24944640 -> 24948736 00010000:00010000:1.0:1789760759.395606:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354546e52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.395618:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354546e52 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.395627:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354546e52 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.395636:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760759.395641:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354546e52 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.395650:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354546e52 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.395669:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354546e52 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.395676:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354546e52 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.395759:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495861376:12345-192.168.203.9@tcp:4:dd.0 Request processed in 24905us (25283us total) trans 4294973407 rc 0/0 00000100:00100000:1.0:1789760759.395770:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6119 00000100:00100000:1.0:1789760759.395780:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743661252240 time 743691554193 state 0x2 ref 0 00000001:00080000:3.0:1789760759.398670:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973407 is committed 00000100:00100000:1.0:1789760759.398788:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760759.398861:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495861504 00000100:00100000:0.0:1789760759.398869:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495861504 00000100:00100000:0.0:1789760759.398935:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743661252240 time 743691554193 state 0x0 ref 1 00000100:00100000:0.0:1789760759.398950:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743691814609 time 743691554193 state 0x1 ref 1 00000100:00100000:0.0:1789760759.398974:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743691814609 time 743719691425 state 0x0 ref 1 00000100:00100000:0.0:1789760759.398987:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04577b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495861504:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760759.399031:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24948736, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc007 00010000:00010000:0.0:1789760759.399040:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546e59 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.399056:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760759.399060:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.399067:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546e59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.399085:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546e59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760759.448378:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24948736 -> 24952832 00010000:00010000:1.0:1789760759.448398:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546e59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.448423:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546e59 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.448439:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546e59 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.448447:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760759.448452:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546e59 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.448462:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546e59 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.448479:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546e59 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.448487:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354546e59 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.448571:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04577b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495861504:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49582us (49782us total) trans 4294973408 rc 0/0 00000100:00100000:1.0:1789760759.448581:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6120 00000100:00100000:1.0:1789760759.448591:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743691814609 time 743719691425 state 0x2 ref 0 00000001:00080000:3.0:1789760759.450743:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973408 is committed 00000100:00100000:1.0:1789760759.452034:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760759.452125:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495861632 00000100:00100000:2.0:1789760759.452135:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495861632 00000100:00100000:2.0:1789760759.452186:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743691814609 time 743719691425 state 0x0 ref 1 00000100:00100000:2.0:1789760759.452202:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743719951841 time 743719691425 state 0x1 ref 1 00000100:00100000:2.0:1789760759.452267:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743719951841 time 743772983223 state 0x0 ref 1 00000100:00100000:2.0:1789760759.452306:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495861632:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760759.452350:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24952832, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc007 00010000:00010000:2.0:1789760759.452359:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546e60 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.452373:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:2.0:1789760759.452377:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.452384:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546e60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.452397:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546e60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.478961:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973409 is committed 00002000:00010000:0.0:1789760759.479211:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24952832 -> 24956928 00010000:00010000:0.0:1789760759.479354:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546e60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.479368:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546e60 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.479375:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546e60 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.479380:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760759.479384:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546e60 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.479389:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546e60 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.479404:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546e60 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.479408:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546e60 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.479513:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495861632:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27232us (27482us total) trans 4294973409 rc 0/0 00000100:00100000:0.0:1789760759.479520:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6121 00000100:00100000:0.0:1789760759.479528:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743719951841 time 743772983223 state 0x2 ref 0 00000100:00100000:0.0:1789760759.483545:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760759.483644:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495861760 00000100:00100000:1.0:1789760759.483654:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495861760 00000100:00100000:1.0:1789760759.483706:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743719951841 time 743772983223 state 0x0 ref 1 00000100:00100000:1.0:1789760759.483725:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743773243639 time 743772983223 state 0x1 ref 1 00000100:00100000:1.0:1789760759.483781:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743773243639 time 743804497585 state 0x0 ref 1 00000100:00100000:1.0:1789760759.483793:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495861760:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760759.483839:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24956928, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc007 00010000:00010000:1.0:1789760759.483848:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546e67 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.483864:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:1.0:1789760759.483868:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.483875:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546e67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.483888:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546e67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760759.508389:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973410 is committed 00002000:00010000:3.0:1789760759.508391:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24956928 -> 24961024 00010000:00010000:3.0:1789760759.508406:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546e67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.508419:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546e67 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.508429:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546e67 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.508436:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760759.508440:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546e67 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.508447:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546e67 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.508462:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546e67 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.508467:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354546e67 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.508576:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495861760:12345-192.168.203.9@tcp:4:dd.0 Request processed in 24781us (25036us total) trans 4294973410 rc 0/0 00000100:00100000:3.0:1789760759.508584:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6122 00000100:00100000:3.0:1789760759.508591:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743773243639 time 743804497585 state 0x2 ref 0 00000100:00100000:1.0:1789760759.512376:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760759.512571:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495861888 00000100:00100000:0.0:1789760759.512647:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495861888 00000100:00100000:0.0:1789760759.512707:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743773243639 time 743804497585 state 0x0 ref 1 00000100:00100000:0.0:1789760759.512724:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743804758001 time 743804497585 state 0x1 ref 1 00000100:00100000:0.0:1789760759.512752:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743804758001 time 743833468566 state 0x0 ref 1 00000100:00100000:0.0:1789760759.512764:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495861888:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760759.512814:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24961024, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc007 00010000:00010000:0.0:1789760759.512823:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546e6e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.512839:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760759.512844:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.512851:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546e6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.512868:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546e6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760759.557759:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973411 is committed 00002000:00010000:2.0:1789760759.558021:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24961024 -> 24965120 00010000:00010000:2.0:1789760759.558038:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546e6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.558051:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546e6e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.558061:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546e6e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.558068:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760759.558191:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546e6e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.558200:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546e6e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.558221:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546e6e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.558229:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354546e6e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.558360:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495861888:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45594us (45989us total) trans 4294973411 rc 0/0 00000100:00100000:2.0:1789760759.558370:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6123 00000100:00100000:2.0:1789760759.558380:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743804758001 time 743833468566 state 0x2 ref 0 00000100:00100000:1.0:1789760759.563100:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760759.563181:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495862016 00000100:00100000:0.0:1789760759.563190:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495862016 00000100:00100000:0.0:1789760759.563245:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743804758001 time 743833468566 state 0x0 ref 1 00000100:00100000:0.0:1789760759.563259:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743833728982 time 743833468566 state 0x1 ref 1 00000100:00100000:0.0:1789760759.563328:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743833728982 time 743884044231 state 0x0 ref 1 00000100:00100000:0.0:1789760759.563341:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903639b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495862016:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760759.563387:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24965120, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc007 00010000:00010000:0.0:1789760759.563396:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546e75 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.563410:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760759.563414:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.563420:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546e75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.563434:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546e75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760759.609034:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973412 is committed 00002000:00010000:2.0:1789760759.609235:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24965120 -> 24969216 00010000:00010000:2.0:1789760759.609252:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546e75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.609264:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546e75 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.609273:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546e75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.609302:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760759.609307:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546e75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.609320:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546e75 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.609341:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546e75 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.609348:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546e75 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.609435:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903639b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495862016:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46092us (46339us total) trans 4294973412 rc 0/0 00000100:00100000:2.0:1789760759.609446:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6124 00000100:00100000:2.0:1789760759.609456:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743833728982 time 743884044231 state 0x2 ref 0 00000100:00100000:0.0:1789760759.616573:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760759.616992:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495862144 00000100:00100000:3.0:1789760759.617003:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495862144 00000100:00100000:3.0:1789760759.617057:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743833728982 time 743884044231 state 0x0 ref 1 00000100:00100000:3.0:1789760759.617155:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743884304647 time 743884044231 state 0x1 ref 1 00000100:00100000:3.0:1789760759.617190:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743884304647 time 743937907165 state 0x0 ref 1 00000100:00100000:3.0:1789760759.617203:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495862144:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760759.617264:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24969216, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc007 00010000:00010000:3.0:1789760759.617275:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546e7c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.617335:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760759.617340:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.617347:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546e7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.617362:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546e7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760759.639444:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24969216 -> 24973312 00010000:00010000:3.0:1789760759.639463:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546e7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.639475:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546e7c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.639484:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546e7c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.639492:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760759.639497:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546e7c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.639505:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546e7c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.639525:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546e7c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.639534:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546e7c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.639621:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495862144:12345-192.168.203.9@tcp:4:dd.0 Request processed in 22413us (23049us total) trans 4294973413 rc 0/0 00000100:00100000:3.0:1789760759.639632:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6125 00000100:00100000:3.0:1789760759.639643:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743884304647 time 743937907165 state 0x2 ref 0 00000001:00080000:0.0:1789760759.640129:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973413 is committed 00000100:00100000:1.0:1789760759.643719:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760759.644101:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495862272 00000100:00100000:2.0:1789760759.644111:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495862272 00000100:00100000:2.0:1789760759.644169:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743884304647 time 743937907165 state 0x0 ref 1 00000100:00100000:2.0:1789760759.644184:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743938167581 time 743937907165 state 0x1 ref 1 00000100:00100000:2.0:1789760759.644213:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743938167581 time 743964929909 state 0x0 ref 1 00000100:00100000:2.0:1789760759.644225:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495862272:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760759.644404:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24973312, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc007 00010000:00010000:2.0:1789760759.644415:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546e83 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.644431:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:2.0:1789760759.644436:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.644442:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546e83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.644456:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546e83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.646355:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760759.670380:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973414 is committed 00002000:00010000:2.0:1789760759.670513:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24973312 -> 24977408 00002000:00010000:2.0:1789760759.670517:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760758 -> 1789760759 00002000:00010000:2.0:1789760759.670520:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760758 -> 1789760759 00010000:00010000:2.0:1789760759.670533:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546e83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.670556:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546e83 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.670566:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546e83 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.670573:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760759.670579:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546e83 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.670588:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546e83 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.670607:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546e83 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.670616:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546e83 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.670707:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495862272:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26478us (26991us total) trans 4294973414 rc 0/0 00000100:00100000:2.0:1789760759.670717:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6126 00000100:00100000:2.0:1789760759.670728:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743938167581 time 743964929909 state 0x2 ref 0 00000100:00100000:1.0:1789760759.675817:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760759.675877:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495862400 00000100:00100000:2.0:1789760759.675887:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495862400 00000100:00100000:2.0:1789760759.675951:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743938167581 time 743964929909 state 0x0 ref 1 00000100:00100000:2.0:1789760759.675968:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743965190325 time 743964929909 state 0x1 ref 1 00000100:00100000:2.0:1789760759.675997:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743965190325 time 743996713635 state 0x0 ref 1 00000100:00100000:2.0:1789760759.676009:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04575500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495862400:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760759.676055:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24977408, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc007 00010000:00010000:2.0:1789760759.676064:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546e8a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.676080:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:2.0:1789760759.676085:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.676091:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546e8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.676106:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546e8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.732270:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973415 is committed 00002000:00010000:1.0:1789760759.732594:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24977408 -> 24981504 00010000:00010000:1.0:1789760759.732610:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546e8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.732623:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546e8a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.732633:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546e8a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.732641:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760759.732645:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546e8a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.732655:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546e8a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.732676:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546e8a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.732684:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354546e8a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.732815:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04575500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495862400:12345-192.168.203.9@tcp:4:dd.0 Request processed in 56802us (57001us total) trans 4294973415 rc 0/0 00000100:00100000:1.0:1789760759.732826:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6127 00000100:00100000:1.0:1789760759.732836:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743965190325 time 743996713635 state 0x2 ref 0 00000100:00100000:3.0:1789760759.738052:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760759.738228:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495862528 00000100:00100000:1.0:1789760759.738238:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495862528 00000100:00100000:1.0:1789760759.738307:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743965190325 time 743996713635 state 0x0 ref 1 00000100:00100000:1.0:1789760759.738322:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743996974051 time 743996713635 state 0x1 ref 1 00000100:00100000:1.0:1789760759.738349:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743996974051 time 744059066193 state 0x0 ref 1 00000100:00100000:1.0:1789760759.738361:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495862528:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760759.738423:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24981504, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc007 00010000:00010000:1.0:1789760759.738432:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546e91 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.738447:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760759.738452:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.738458:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546e91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.738471:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546e91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760759.789645:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973416 is committed 00002000:00010000:3.0:1789760759.790753:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24981504 -> 24985600 00010000:00010000:3.0:1789760759.790772:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546e91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.790784:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546e91 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.790793:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546e91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.790800:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760759.790805:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546e91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.790814:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546e91 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.790835:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546e91 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.790844:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546e91 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.790958:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495862528:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52595us (52912us total) trans 4294973416 rc 0/0 00000100:00100000:3.0:1789760759.790969:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6128 00000100:00100000:3.0:1789760759.790978:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743996974051 time 744059066193 state 0x2 ref 0 00000100:00100000:3.0:1789760759.796948:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760759.798006:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495862656 00000100:00100000:2.0:1789760759.798019:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495862656 00000100:00100000:2.0:1789760759.798080:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 743996974051 time 744059066193 state 0x0 ref 1 00000100:00100000:2.0:1789760759.798097:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744059326609 time 744059066193 state 0x1 ref 1 00000100:00100000:2.0:1789760759.798129:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744059326609 time 744118845368 state 0x0 ref 1 00000100:00100000:2.0:1789760759.798142:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495862656:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760759.798191:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24985600, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc007 00010000:00010000:2.0:1789760759.798200:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546e98 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.798216:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760759.798221:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.798516:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546e98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.798532:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546e98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760759.832900:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973417 is committed 00002000:00010000:2.0:1789760759.833649:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24985600 -> 24989696 00010000:00010000:2.0:1789760759.833665:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546e98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.833677:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546e98 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.833687:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546e98 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.833693:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760759.833698:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546e98 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.833706:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546e98 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.833726:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546e98 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.833733:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546e98 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.833861:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495862656:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35717us (36917us total) trans 4294973417 rc 0/0 00000100:00100000:2.0:1789760759.833875:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6129 00000100:00100000:2.0:1789760759.833895:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744059326609 time 744118845368 state 0x2 ref 0 00000100:00100000:1.0:1789760759.840592:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760759.840691:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495862784 00000100:00100000:2.0:1789760759.840704:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495862784 00000100:00100000:2.0:1789760759.840770:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744059326609 time 744118845368 state 0x0 ref 1 00000100:00100000:2.0:1789760759.840789:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744119105784 time 744118845368 state 0x1 ref 1 00000100:00100000:2.0:1789760759.840823:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744119105784 time 744161539741 state 0x0 ref 1 00000100:00100000:2.0:1789760759.840838:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04577b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495862784:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760759.840902:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24989696, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc007 00010000:00010000:2.0:1789760759.840919:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546e9f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.840934:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760759.840938:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.840944:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546e9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.840957:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546e9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760759.888565:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24989696 -> 24993792 00010000:00010000:2.0:1789760759.888587:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546e9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760759.888600:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973418 is committed 00010000:00010000:2.0:1789760759.888600:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546e9f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.888609:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546e9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.888616:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760759.888769:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546e9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.888780:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546e9f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.888800:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546e9f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.888808:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546e9f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.888916:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04577b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495862784:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48067us (48321us total) trans 4294973418 rc 0/0 00000100:00100000:2.0:1789760759.888926:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6130 00000100:00100000:2.0:1789760759.888936:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744119105784 time 744161539741 state 0x2 ref 0 00000100:00100000:0.0:1789760759.894382:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760759.895080:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495862912 00000100:00100000:3.0:1789760759.895091:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495862912 00000100:00100000:3.0:1789760759.895152:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744119105784 time 744161539741 state 0x0 ref 1 00000100:00100000:3.0:1789760759.895169:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744161800157 time 744161539741 state 0x1 ref 1 00000100:00100000:3.0:1789760759.895202:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744161800157 time 744215919138 state 0x0 ref 1 00000100:00100000:3.0:1789760759.895215:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495862912:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760759.895270:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24993792, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc007 00010000:00010000:3.0:1789760759.895309:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ea6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.895328:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760759.895333:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.895340:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ea6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.895354:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ea6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760759.932497:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24993792 -> 24997888 00010000:00010000:1.0:1789760759.932521:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ea6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.932534:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ea6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.932544:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ea6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.932557:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760759.932562:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ea6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.932571:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ea6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.932592:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ea6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.932600:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354546ea6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.932711:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495862912:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37492us (38333us total) trans 4294973419 rc 0/0 00000100:00100000:1.0:1789760759.932723:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6131 00000100:00100000:1.0:1789760759.932735:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744161800157 time 744215919138 state 0x2 ref 0 00000001:00080000:2.0:1789760759.933175:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973419 is committed 00000100:00100000:3.0:1789760759.941323:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760759.942317:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495863040 00000100:00100000:2.0:1789760759.942329:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495863040 00000100:00100000:0.0:1789760759.942470:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744161800157 time 744215919138 state 0x0 ref 1 00000100:00100000:0.0:1789760759.942485:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744216179554 time 744215919138 state 0x1 ref 1 00000100:00100000:0.0:1789760759.942517:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744216179554 time 744263233465 state 0x0 ref 1 00000100:00100000:0.0:1789760759.942531:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495863040:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760759.942585:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24997888, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc007 00010000:00010000:0.0:1789760759.942594:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546ead lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.942612:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760759.942617:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.942623:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546ead lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.942637:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546ead lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.999183:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973420 is committed 00002000:00010000:0.0:1789760759.999529:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24997888 -> 25001984 00010000:00010000:0.0:1789760759.999545:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546ead lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.999557:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546ead lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.999567:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546ead lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.999574:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760759.999579:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546ead lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.999588:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546ead lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.999607:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546ead lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.999616:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354546ead lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.999741:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495863040:12345-192.168.203.9@tcp:4:dd.0 Request processed in 57209us (58466us total) trans 4294973420 rc 0/0 00000100:00100000:0.0:1789760759.999755:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6132 00000100:00100000:0.0:1789760759.999764:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744216179554 time 744263233465 state 0x2 ref 0 00000100:00100000:0.0:1789760760.007598:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760760.007828:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495863168 00000100:00100000:2.0:1789760760.007839:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495863168 00000100:00100000:2.0:1789760760.007919:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744216179554 time 744263233465 state 0x0 ref 1 00000100:00100000:2.0:1789760760.007941:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744263493881 time 744263233465 state 0x1 ref 1 00000100:00100000:2.0:1789760760.008010:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744263493881 time 744328726497 state 0x0 ref 1 00000100:00100000:2.0:1789760760.008023:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495863168:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760760.008069:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25001984, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc007 00010000:00010000:2.0:1789760760.008078:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546eb4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.008094:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760760.008099:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.008106:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546eb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.008120:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546eb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760760.050003:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25001984 -> 25006080 00010000:00010000:0.0:1789760760.050023:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546eb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.050035:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546eb4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.050045:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546eb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.050053:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760760.050058:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546eb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.050067:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546eb4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.050087:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546eb4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.050096:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546eb4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.050186:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495863168:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42161us (42592us total) trans 4294973421 rc 0/0 00000100:00100000:0.0:1789760760.050197:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6133 00000100:00100000:0.0:1789760760.050208:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744263493881 time 744328726497 state 0x2 ref 0 00000001:00080000:3.0:1789760760.050408:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973421 is committed 00000100:00100000:2.0:1789760760.053659:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760760.053757:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495863296 00000100:00100000:1.0:1789760760.053767:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495863296 00000100:00100000:1.0:1789760760.053821:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744263493881 time 744328726497 state 0x0 ref 1 00000100:00100000:1.0:1789760760.053835:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744328986913 time 744328726497 state 0x1 ref 1 00000100:00100000:1.0:1789760760.053941:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744328986913 time 744374657757 state 0x0 ref 1 00000100:00100000:1.0:1789760760.053958:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495863296:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760760.054006:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25006080, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc007 00010000:00010000:1.0:1789760760.054016:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ebb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.054032:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760760.054037:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.054043:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ebb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.054057:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ebb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760760.093056:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25006080 -> 25010176 00010000:00010000:1.0:1789760760.093078:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ebb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.093092:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ebb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.093102:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ebb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.093110:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760760.093116:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ebb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.093128:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ebb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.093150:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ebb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.093159:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ebb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.093379:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495863296:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39418us (39723us total) trans 4294973422 rc 0/0 00000100:00100000:1.0:1789760760.093390:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6134 00000100:00100000:1.0:1789760760.093399:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744328986913 time 744374657757 state 0x2 ref 0 00000001:00080000:3.0:1789760760.093682:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973422 is committed 00000100:00100000:2.0:1789760760.107680:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760760.107779:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495863424 00000100:00100000:0.0:1789760760.107788:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495863424 00000100:00100000:0.0:1789760760.107924:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744328986913 time 744374657757 state 0x0 ref 1 00000100:00100000:0.0:1789760760.110750:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744374918173 time 744374657757 state 0x1 ref 1 00000100:00100000:0.0:1789760760.110869:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744374918173 time 744431584370 state 0x0 ref 1 00000100:00100000:0.0:1789760760.110893:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495863424:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760760.110958:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25010176, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc007 00010000:00010000:0.0:1789760760.110968:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546ec2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.110984:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760760.110989:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760760.110996:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546ec2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.111010:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546ec2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760760.148538:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973423 is committed 00002000:00010000:0.0:1789760760.148872:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25010176 -> 25014272 00010000:00010000:0.0:1789760760.148953:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546ec2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.148967:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546ec2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.148977:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546ec2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.148984:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760760.148988:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546ec2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.148996:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546ec2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.149015:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546ec2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.149023:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546ec2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.149143:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495863424:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38258us (41468us total) trans 4294973423 rc 0/0 00000100:00100000:0.0:1789760760.149155:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6135 00000100:00100000:0.0:1789760760.149164:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744374918173 time 744431584370 state 0x2 ref 0 00000100:00100000:1.0:1789760760.153806:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760760.153911:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495863552 00000100:00100000:2.0:1789760760.153921:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495863552 00000100:00100000:2.0:1789760760.153984:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744374918173 time 744431584370 state 0x0 ref 1 00000100:00100000:2.0:1789760760.154000:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744431844786 time 744431584370 state 0x1 ref 1 00000100:00100000:2.0:1789760760.154029:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744431844786 time 744474745487 state 0x0 ref 1 00000100:00100000:2.0:1789760760.154041:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495863552:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760760.154088:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25014272, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc007 00010000:00010000:2.0:1789760760.154098:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546ec9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.154114:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760760.154118:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.154126:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546ec9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.154139:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546ec9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760760.189532:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973424 is committed 00002000:00010000:0.0:1789760760.189629:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25014272 -> 25018368 00010000:00010000:0.0:1789760760.189645:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546ec9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.189659:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546ec9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.189668:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546ec9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.189675:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760760.189680:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546ec9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.189689:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546ec9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.189709:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546ec9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.189716:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546ec9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.189895:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495863552:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35843us (36085us total) trans 4294973424 rc 0/0 00000100:00100000:0.0:1789760760.189915:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6136 00000100:00100000:0.0:1789760760.189925:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744431844786 time 744474745487 state 0x2 ref 0 00000100:00100000:2.0:1789760760.194856:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760760.195028:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495863680 00000100:00100000:1.0:1789760760.195039:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495863680 00000100:00100000:1.0:1789760760.195090:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744431844786 time 744474745487 state 0x0 ref 1 00000100:00100000:1.0:1789760760.195105:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744475005903 time 744474745487 state 0x1 ref 1 00000100:00100000:1.0:1789760760.195136:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744475005903 time 744515853109 state 0x0 ref 1 00000100:00100000:1.0:1789760760.195150:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495863680:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760760.195196:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25018368, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc007 00010000:00010000:1.0:1789760760.195206:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ed0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.195222:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760760.195227:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.195234:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ed0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.195247:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ed0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760760.229363:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973425 is committed 00002000:00010000:0.0:1789760760.229958:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25018368 -> 25022464 00010000:00010000:0.0:1789760760.230028:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ed0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.230042:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ed0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.230051:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ed0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.230059:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760760.230065:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ed0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.230074:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ed0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.230093:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ed0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.230106:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546ed0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.230788:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495863680:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35634us (35933us total) trans 4294973425 rc 0/0 00000100:00100000:0.0:1789760760.230798:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6137 00000100:00100000:0.0:1789760760.230808:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744475005903 time 744515853109 state 0x2 ref 0 00000100:00100000:3.0:1789760760.235751:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760760.236368:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495863808 00000100:00100000:0.0:1789760760.236378:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495863808 00000100:00100000:0.0:1789760760.236432:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744475005903 time 744515853109 state 0x0 ref 1 00000100:00100000:0.0:1789760760.236482:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744516113525 time 744515853109 state 0x1 ref 1 00000100:00100000:0.0:1789760760.236513:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744516113525 time 744557229338 state 0x0 ref 1 00000100:00100000:0.0:1789760760.236527:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495863808:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760760.236573:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25022464, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc007 00010000:00010000:0.0:1789760760.236582:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546ed7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.236597:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760760.236601:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760760.236608:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546ed7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.236626:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546ed7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760760.274924:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25022464 -> 25026560 00010000:00010000:1.0:1789760760.274944:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546ed7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.274957:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546ed7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.274966:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546ed7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.274973:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760760.274978:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546ed7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.274987:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546ed7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.275009:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546ed7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.275016:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354546ed7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.275120:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495863808:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38591us (39373us total) trans 4294973426 rc 0/0 00000100:00100000:1.0:1789760760.275131:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6138 00000100:00100000:1.0:1789760760.275141:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744516113525 time 744557229338 state 0x2 ref 0 00000001:00080000:3.0:1789760760.275338:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973426 is committed 00000100:00100000:2.0:1789760760.280116:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760760.280478:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495863936 00000100:00100000:0.0:1789760760.280488:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495863936 00000100:00100000:0.0:1789760760.280537:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744516113525 time 744557229338 state 0x0 ref 1 00000100:00100000:0.0:1789760760.280551:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744557489754 time 744557229338 state 0x1 ref 1 00000100:00100000:0.0:1789760760.280580:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744557489754 time 744601297259 state 0x0 ref 1 00000100:00100000:0.0:1789760760.280592:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495863936:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760760.280648:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25026560, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc007 00010000:00010000:0.0:1789760760.280657:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546ede lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.280675:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760760.280679:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760760.280685:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546ede lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.280699:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546ede lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760760.327993:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25026560 -> 25030656 00010000:00010000:0.0:1789760760.328013:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546ede lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.328026:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546ede lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.328034:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546ede lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.328041:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760760.328045:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546ede lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.328054:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546ede lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.328072:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546ede lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.328080:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546ede lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.328221:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495863936:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47624us (48108us total) trans 4294973427 rc 0/0 00000100:00100000:0.0:1789760760.328232:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6139 00000100:00100000:0.0:1789760760.328242:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744557489754 time 744601297259 state 0x2 ref 0 00000001:00080000:2.0:1789760760.328558:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973427 is committed 00000100:00100000:0.0:1789760760.337496:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760760.337600:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495864064 00000100:00100000:1.0:1789760760.337610:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495864064 00000100:00100000:1.0:1789760760.337733:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744557489754 time 744601297259 state 0x0 ref 1 00000100:00100000:1.0:1789760760.337749:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744601557675 time 744601297259 state 0x1 ref 1 00000100:00100000:1.0:1789760760.337783:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744601557675 time 744658500406 state 0x0 ref 1 00000100:00100000:1.0:1789760760.337797:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495864064:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760760.337847:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25030656, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc007 00010000:00010000:1.0:1789760760.337858:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546ee5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.337873:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760760.337878:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.337886:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546ee5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.337912:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546ee5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760760.380520:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25030656 -> 25034752 00010000:00010000:0.0:1789760760.380540:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546ee5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.380552:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546ee5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.380561:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546ee5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.380568:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760760.380572:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546ee5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.380582:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546ee5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.380601:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546ee5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.380609:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354546ee5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.380711:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495864064:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42912us (43437us total) trans 4294973428 rc 0/0 00000100:00100000:0.0:1789760760.380726:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6140 00000100:00100000:0.0:1789760760.380736:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744601557675 time 744658500406 state 0x2 ref 0 00000001:00080000:2.0:1789760760.382207:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973428 is committed 00000100:00100000:1.0:1789760760.386359:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760760.386414:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495864192 00000100:00100000:0.0:1789760760.386425:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495864192 00000100:00100000:0.0:1789760760.386480:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744601557675 time 744658500406 state 0x0 ref 1 00000100:00100000:0.0:1789760760.386495:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744658760822 time 744658500406 state 0x1 ref 1 00000100:00100000:0.0:1789760760.386529:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744658760822 time 744707245450 state 0x0 ref 1 00000100:00100000:0.0:1789760760.386542:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495864192:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760760.386589:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25034752, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc007 00010000:00010000:0.0:1789760760.386598:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546eec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.386614:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760760.386628:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760760.386634:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546eec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.386649:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546eec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760760.420399:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25034752 -> 25038848 00010000:00010000:0.0:1789760760.420420:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546eec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.420433:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546eec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.420442:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546eec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.420450:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760760.420456:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546eec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.420465:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546eec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.420487:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546eec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.420494:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354546eec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.420592:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495864192:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34046us (34235us total) trans 4294973429 rc 0/0 00000100:00100000:0.0:1789760760.420602:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6141 00000100:00100000:0.0:1789760760.420612:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744658760822 time 744707245450 state 0x2 ref 0 00000001:00080000:2.0:1789760760.420911:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973429 is committed 00000100:00100000:1.0:1789760760.425978:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760760.427016:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495864320 00000100:00100000:2.0:1789760760.427029:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495864320 00000100:00100000:2.0:1789760760.427088:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744658760822 time 744707245450 state 0x0 ref 1 00000100:00100000:2.0:1789760760.427108:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744707505866 time 744707245450 state 0x1 ref 1 00000100:00100000:2.0:1789760760.427138:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744707505866 time 744747855128 state 0x0 ref 1 00000100:00100000:2.0:1789760760.427151:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04291500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495864320:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760760.427210:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25038848, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc007 00010000:00010000:2.0:1789760760.427220:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546ef3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.427237:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760760.427243:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.427250:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546ef3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.427263:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546ef3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760760.470006:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25038848 -> 25042944 00010000:00010000:2.0:1789760760.470026:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546ef3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.470039:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546ef3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.470049:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546ef3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.470061:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760760.470065:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546ef3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.470074:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546ef3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.470106:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546ef3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.470113:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546ef3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760760.470158:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973430 is committed 00000100:00100000:2.0:1789760760.470203:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04291500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495864320:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43050us (44231us total) trans 4294973430 rc 0/0 00000100:00100000:2.0:1789760760.470214:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6142 00000100:00100000:2.0:1789760760.470223:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744707505866 time 744747855128 state 0x2 ref 0 00000100:00100000:3.0:1789760760.478519:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760760.478590:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495864448 00000100:00100000:0.0:1789760760.478601:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495864448 00000100:00100000:0.0:1789760760.478657:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744707505866 time 744747855128 state 0x0 ref 1 00000100:00100000:0.0:1789760760.478672:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744748115544 time 744747855128 state 0x1 ref 1 00000100:00100000:0.0:1789760760.478742:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744748115544 time 744799458114 state 0x0 ref 1 00000100:00100000:0.0:1789760760.478756:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495864448:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760760.478804:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25042944, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc007 00010000:00010000:0.0:1789760760.478813:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546efa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.478829:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760760.478833:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760760.478839:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546efa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.478854:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546efa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760760.522757:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25042944 -> 25047040 00010000:00010000:0.0:1789760760.522778:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546efa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.522791:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546efa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.522807:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546efa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.522815:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760760.522820:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546efa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.522828:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546efa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.522846:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546efa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.522854:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354546efa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.523006:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495864448:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44247us (44492us total) trans 4294973431 rc 0/0 00000100:00100000:0.0:1789760760.523017:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6143 00000100:00100000:0.0:1789760760.523026:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744748115544 time 744799458114 state 0x2 ref 0 00000001:00080000:2.0:1789760760.523135:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973431 is committed 00000100:00100000:0.0:1789760760.526266:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760760.526512:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495864576 00000100:00100000:1.0:1789760760.526523:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495864576 00000100:00100000:1.0:1789760760.526576:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744748115544 time 744799458114 state 0x0 ref 1 00000100:00100000:1.0:1789760760.526591:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744799718530 time 744799458114 state 0x1 ref 1 00000100:00100000:1.0:1789760760.526623:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744799718530 time 744847339273 state 0x0 ref 1 00000100:00100000:1.0:1789760760.526636:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495864576:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760760.526682:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25047040, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc007 00010000:00010000:1.0:1789760760.526691:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546f01 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.526707:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760760.526712:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.526719:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546f01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.526739:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546f01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760760.577257:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973432 is committed 00002000:00010000:3.0:1789760760.577412:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25047040 -> 25051136 00010000:00010000:3.0:1789760760.577427:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546f01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.577440:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546f01 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.577449:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546f01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.577456:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760760.577461:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546f01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.577471:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546f01 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.577491:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546f01 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.577499:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546f01 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.577599:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495864576:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50959us (51335us total) trans 4294973432 rc 0/0 00000100:00100000:3.0:1789760760.577610:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6144 00000100:00100000:3.0:1789760760.577622:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744799718530 time 744847339273 state 0x2 ref 0 00000100:00100000:3.0:1789760760.587199:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760760.587386:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495864704 00000100:00100000:2.0:1789760760.587397:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495864704 00000100:00100000:2.0:1789760760.587451:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744799718530 time 744847339273 state 0x0 ref 1 00000100:00100000:2.0:1789760760.587465:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744847599689 time 744847339273 state 0x1 ref 1 00000100:00100000:2.0:1789760760.587501:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744847599689 time 744908217410 state 0x0 ref 1 00000100:00100000:2.0:1789760760.587514:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495864704:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760760.587561:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25051136, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc007 00010000:00010000:2.0:1789760760.587570:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546f08 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.587587:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760760.587591:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.587598:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546f08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.587612:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546f08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760760.652152:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973433 is committed 00002000:00010000:0.0:1789760760.652435:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25051136 -> 25055232 00010000:00010000:0.0:1789760760.652451:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546f08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.652464:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546f08 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.652474:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546f08 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.652482:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760760.652486:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546f08 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.652495:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546f08 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.652514:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546f08 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.652522:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546f08 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.652648:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495864704:12345-192.168.203.9@tcp:4:dd.0 Request processed in 65131us (65455us total) trans 4294973433 rc 0/0 00000100:00100000:0.0:1789760760.652658:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6145 00000100:00100000:0.0:1789760760.652668:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744847599689 time 744908217410 state 0x2 ref 0 00000100:00100000:0.0:1789760760.657707:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760760.657824:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495864832 00000100:00100000:1.0:1789760760.657833:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495864832 00000100:00100000:1.0:1789760760.657967:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744847599689 time 744908217410 state 0x0 ref 1 00000100:00100000:1.0:1789760760.657984:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744908477826 time 744908217410 state 0x1 ref 1 00000100:00100000:1.0:1789760760.658016:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744908477826 time 744978732796 state 0x0 ref 1 00000100:00100000:1.0:1789760760.658029:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495864832:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760760.658077:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25055232, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc007 00010000:00010000:1.0:1789760760.658086:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546f0f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.658102:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760760.658106:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.658113:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546f0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.658489:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546f0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.671075:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760760.689269:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973434 is committed 00002000:00010000:0.0:1789760760.689435:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25055232 -> 25059328 00002000:00010000:0.0:1789760760.689440:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760759 -> 1789760760 00002000:00010000:0.0:1789760760.689443:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760759 -> 1789760760 00010000:00010000:0.0:1789760760.689457:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546f0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.689469:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546f0f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.689479:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546f0f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.689668:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760760.689674:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546f0f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.689683:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546f0f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.689705:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546f0f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.689714:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546f0f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.689803:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495864832:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31772us (32101us total) trans 4294973434 rc 0/0 00000100:00100000:0.0:1789760760.689817:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6146 00000100:00100000:0.0:1789760760.689827:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744908477826 time 744978732796 state 0x2 ref 0 00000100:00100000:2.0:1789760760.695786:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760760.697435:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495864960 00000100:00100000:3.0:1789760760.697449:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495864960 00000100:00100000:3.0:1789760760.697509:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744908477826 time 744978732796 state 0x0 ref 1 00000100:00100000:3.0:1789760760.697524:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744978993212 time 744978732796 state 0x1 ref 1 00000100:00100000:3.0:1789760760.697622:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744978993212 time 745018337560 state 0x0 ref 1 00000100:00100000:3.0:1789760760.697637:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495864960:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760760.697694:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25059328, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc007 00010000:00010000:3.0:1789760760.697703:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546f16 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.697720:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:3.0:1789760760.697725:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.697732:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546f16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.697747:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546f16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760760.737132:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973435 is committed 00002000:00010000:1.0:1789760760.737235:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25059328 -> 25063424 00010000:00010000:1.0:1789760760.737252:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546f16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.737264:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546f16 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.737272:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546f16 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.737298:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760760.737304:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546f16 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.737313:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546f16 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.737332:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546f16 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.737340:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354546f16 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.737450:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495864960:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39811us (41668us total) trans 4294973435 rc 0/0 00000100:00100000:1.0:1789760760.737460:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6147 00000100:00100000:1.0:1789760760.737477:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744978993212 time 745018337560 state 0x2 ref 0 00000100:00100000:3.0:1789760760.746381:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760760.749830:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495865088 00000100:00100000:0.0:1789760760.749845:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495865088 00000100:00100000:2.0:1789760760.753034:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 744978993212 time 745018337560 state 0x0 ref 1 00000100:00100000:2.0:1789760760.753053:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745018597976 time 745018337560 state 0x1 ref 1 00000100:00100000:2.0:1789760760.753149:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745018597976 time 745073865581 state 0x0 ref 1 00000100:00100000:2.0:1789760760.753165:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495865088:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760760.753216:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25063424, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc007 00010000:00010000:2.0:1789760760.753226:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f1d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.753243:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760760.753247:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.753252:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.753265:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760760.804535:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973436 is committed 00002000:00010000:2.0:1789760760.804697:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25063424 -> 25067520 00010000:00010000:2.0:1789760760.804716:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.804729:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f1d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.804738:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f1d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.804745:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760760.804750:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f1d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.804771:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f1d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.804797:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f1d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.804805:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f1d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.804945:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495865088:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51780us (58569us total) trans 4294973436 rc 0/0 00000100:00100000:2.0:1789760760.804956:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6148 00000100:00100000:2.0:1789760760.804966:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745018597976 time 745073865581 state 0x2 ref 0 00000100:00100000:0.0:1789760760.808658:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760760.808918:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495865216 00000100:00100000:1.0:1789760760.808927:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495865216 00000100:00100000:1.0:1789760760.808984:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745018597976 time 745073865581 state 0x0 ref 1 00000100:00100000:1.0:1789760760.808998:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745074125997 time 745073865581 state 0x1 ref 1 00000100:00100000:1.0:1789760760.809032:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745074125997 time 745129748909 state 0x0 ref 1 00000100:00100000:1.0:1789760760.809044:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495865216:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760760.809090:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25067520, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc007 00010000:00010000:1.0:1789760760.809099:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f24 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.809114:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760760.809119:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.809125:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.809138:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760760.867158:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25067520 -> 25071616 00010000:00010000:1.0:1789760760.867176:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.867189:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f24 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.867200:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.867322:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760760.867328:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.867338:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f24 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.867358:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f24 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.867366:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f24 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.867449:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495865216:12345-192.168.203.9@tcp:4:dd.0 Request processed in 58402us (58794us total) trans 4294973437 rc 0/0 00000100:00100000:1.0:1789760760.867460:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6149 00000100:00100000:1.0:1789760760.867469:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745074125997 time 745129748909 state 0x2 ref 0 00000001:00080000:0.0:1789760760.868791:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973437 is committed 00000100:00100000:3.0:1789760760.871918:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760760.872308:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495865344 00000100:00100000:1.0:1789760760.872318:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495865344 00000100:00100000:1.0:1789760760.872373:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745074125997 time 745129748909 state 0x0 ref 1 00000100:00100000:1.0:1789760760.872389:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745130009325 time 745129748909 state 0x1 ref 1 00000100:00100000:1.0:1789760760.872418:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745130009325 time 745193134833 state 0x0 ref 1 00000100:00100000:1.0:1789760760.872430:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495865344:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760760.872477:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25071616, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc007 00010000:00010000:1.0:1789760760.872485:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546f2b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.872501:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760760.872505:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.872511:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546f2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.872524:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546f2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760760.902837:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25071616 -> 25075712 00010000:00010000:3.0:1789760760.902855:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546f2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.902868:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546f2b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.902887:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546f2b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.902903:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760760.902909:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546f2b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.902921:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546f2b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.902940:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546f2b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.902949:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354546f2b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.903046:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495865344:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30613us (31143us total) trans 4294973438 rc 0/0 00000100:00100000:3.0:1789760760.903056:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6150 00000100:00100000:3.0:1789760760.903065:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745130009325 time 745193134833 state 0x2 ref 0 00000001:00080000:2.0:1789760760.903637:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973438 is committed 00000100:00100000:1.0:1789760760.908134:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760760.908808:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495865472 00000100:00100000:2.0:1789760760.908818:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495865472 00000100:00100000:2.0:1789760760.908873:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745130009325 time 745193134833 state 0x0 ref 1 00000100:00100000:2.0:1789760760.908888:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745193395249 time 745193134833 state 0x1 ref 1 00000100:00100000:2.0:1789760760.908929:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745193395249 time 745229645976 state 0x0 ref 1 00000100:00100000:2.0:1789760760.908941:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04291180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495865472:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760760.908985:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25075712, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc007 00010000:00010000:2.0:1789760760.908994:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f32 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.909016:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760760.909021:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.909027:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.909040:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760760.950570:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973439 is committed 00002000:00010000:2.0:1789760760.950676:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25075712 -> 25079808 00010000:00010000:2.0:1789760760.950691:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.950704:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f32 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.950713:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.950721:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760760.950725:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.950735:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f32 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.950754:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f32 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.950763:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f32 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.950910:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04291180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495865472:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41966us (42784us total) trans 4294973439 rc 0/0 00000100:00100000:2.0:1789760760.950921:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6151 00000100:00100000:2.0:1789760760.950930:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745193395249 time 745229645976 state 0x2 ref 0 00000100:00100000:0.0:1789760760.954443:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760760.954539:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495865600 00000100:00100000:1.0:1789760760.954548:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495865600 00000100:00100000:1.0:1789760760.954601:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745193395249 time 745229645976 state 0x0 ref 1 00000100:00100000:1.0:1789760760.954617:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745229906392 time 745229645976 state 0x1 ref 1 00000100:00100000:1.0:1789760760.954651:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745229906392 time 745275368455 state 0x0 ref 1 00000100:00100000:1.0:1789760760.954663:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495865600:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760760.954716:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25079808, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc007 00010000:00010000:1.0:1789760760.954726:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f39 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.954741:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760760.954746:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.954753:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.954767:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760760.988868:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25079808 -> 25083904 00010000:00010000:1.0:1789760760.988909:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.988921:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f39 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.988930:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.988936:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760760.988941:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.988950:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f39 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.988970:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f39 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.988979:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f39 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.989071:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495865600:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34405us (34634us total) trans 4294973440 rc 0/0 00000100:00100000:1.0:1789760760.989082:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6152 00000100:00100000:1.0:1789760760.989092:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745229906392 time 745275368455 state 0x2 ref 0 00000001:00080000:1.0:1789760760.989146:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973440 is committed 00000100:00100000:2.0:1789760760.995651:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760760.995743:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495865728 00000100:00100000:3.0:1789760760.995753:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495865728 00000100:00100000:3.0:1789760760.995811:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745229906392 time 745275368455 state 0x0 ref 1 00000100:00100000:3.0:1789760760.995827:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745275628871 time 745275368455 state 0x1 ref 1 00000100:00100000:3.0:1789760760.995857:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745275628871 time 745316574059 state 0x0 ref 1 00000100:00100000:3.0:1789760760.995869:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495865728:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760760.995934:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25083904, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc007 00010000:00010000:3.0:1789760760.995944:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546f40 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.995960:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:3.0:1789760760.995964:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.995971:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546f40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.995983:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546f40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760761.025606:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973441 is committed 00002000:00010000:3.0:1789760761.025704:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25083904 -> 25088000 00010000:00010000:3.0:1789760761.025719:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546f40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.025732:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546f40 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.025742:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546f40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.025749:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760761.025754:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546f40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.025763:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546f40 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.025782:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546f40 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.025791:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546f40 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.025977:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495865728:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30105us (30330us total) trans 4294973441 rc 0/0 00000100:00100000:3.0:1789760761.025988:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6153 00000100:00100000:3.0:1789760761.025998:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745275628871 time 745316574059 state 0x2 ref 0 00000100:00100000:0.0:1789760761.031271:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760761.031463:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495865856 00000100:00100000:2.0:1789760761.031474:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495865856 00000100:00100000:2.0:1789760761.031545:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745275628871 time 745316574059 state 0x0 ref 1 00000100:00100000:2.0:1789760761.031561:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745316834475 time 745316574059 state 0x1 ref 1 00000100:00100000:2.0:1789760761.031586:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745316834475 time 745352303129 state 0x0 ref 1 00000100:00100000:2.0:1789760761.031598:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f872680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495865856:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760761.031646:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25088000, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc007 00010000:00010000:2.0:1789760761.031656:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f47 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.031673:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760761.031677:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.031684:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.031697:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760761.034344:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760761.073434:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973442 is committed 00002000:00010000:2.0:1789760761.073593:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25088000 -> 25092096 00010000:00010000:2.0:1789760761.073608:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.073621:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f47 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.073630:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f47 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.073637:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760761.073641:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f47 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.073651:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f47 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.073669:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f47 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.073677:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f47 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.073770:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f872680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495865856:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42168us (42503us total) trans 4294973442 rc 0/0 00000100:00100000:2.0:1789760761.073781:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6154 00000100:00100000:2.0:1789760761.073791:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745316834475 time 745352303129 state 0x2 ref 0 00000100:00100000:3.0:1789760761.077049:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760761.077130:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495865984 00000100:00100000:1.0:1789760761.077140:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495865984 00000100:00100000:1.0:1789760761.077192:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745316834475 time 745352303129 state 0x0 ref 1 00000100:00100000:1.0:1789760761.077207:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745352563545 time 745352303129 state 0x1 ref 1 00000100:00100000:1.0:1789760761.077244:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745352563545 time 745397960294 state 0x0 ref 1 00000100:00100000:1.0:1789760761.077256:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495865984:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760761.077323:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25092096, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc007 00010000:00010000:1.0:1789760761.077333:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f4e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.077353:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760761.077357:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.077364:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.077377:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760761.106713:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973443 is committed 00002000:00010000:1.0:1789760761.106822:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25092096 -> 25096192 00010000:00010000:1.0:1789760761.106839:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.106852:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f4e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.106862:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.106869:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760761.106885:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.106904:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f4e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.106923:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f4e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.106931:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354546f4e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.107028:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495865984:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29769us (29983us total) trans 4294973443 rc 0/0 00000100:00100000:1.0:1789760761.107038:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6155 00000100:00100000:1.0:1789760761.107048:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745352563545 time 745397960294 state 0x2 ref 0 00000100:00100000:1.0:1789760761.110993:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760761.111122:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495866112 00000100:00100000:3.0:1789760761.111133:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495866112 00000100:00100000:3.0:1789760761.111187:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745352563545 time 745397960294 state 0x0 ref 1 00000100:00100000:3.0:1789760761.111203:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745398220710 time 745397960294 state 0x1 ref 1 00000100:00100000:3.0:1789760761.111239:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745398220710 time 745431955300 state 0x0 ref 1 00000100:00100000:3.0:1789760761.111251:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495866112:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760761.111326:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25096192, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc007 00010000:00010000:3.0:1789760761.111335:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f55 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.111351:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:3.0:1789760761.111355:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.111362:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.111377:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.118413:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f04929500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520311040:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760761.118740:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760761.118821:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520311040 00000100:00100000:0.0:1789760761.118830:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520311040 00000100:00100000:0.0:1789760761.118905:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 207 00000100:00100000:0.0:1789760761.118915:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520311040:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789760761.118939:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527761408 pending 1703936 free 3833593856 avail 3822059520 00000100:00100000:0.0:1789760761.118983:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520311040:12345-0@lo:13:osp-pre-0-0.0 Request processed in 69us (243us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760761.118989:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 207 00000100:00100000:2.0:1789760761.119341:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f04929500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520311040:0@lo:13:osp-pre-0-0.0 00000100:00080000:2.0:1789760761.119363:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760761.137099:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973444 is committed 00002000:00010000:3.0:1789760761.137200:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25096192 -> 25100288 00010000:00010000:3.0:1789760761.137346:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.137362:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f55 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.137377:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.137385:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760761.137390:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.137400:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f55 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.137419:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f55 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.137427:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f55 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.137519:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495866112:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26264us (26530us total) trans 4294973444 rc 0/0 00000100:00100000:3.0:1789760761.137528:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6156 00000100:00100000:3.0:1789760761.137538:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745398220710 time 745431955300 state 0x2 ref 0 00000100:00100000:3.0:1789760761.140995:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760761.141187:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495866240 00000100:00100000:2.0:1789760761.141197:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495866240 00000100:00100000:2.0:1789760761.141261:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745398220710 time 745431955300 state 0x0 ref 1 00000100:00100000:2.0:1789760761.141413:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745432215716 time 745431955300 state 0x1 ref 1 00000100:00100000:2.0:1789760761.141452:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745432215716 time 745462167988 state 0x0 ref 1 00000100:00100000:2.0:1789760761.141468:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495866240:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760761.141515:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25100288, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc007 00010000:00010000:2.0:1789760761.141525:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f5c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.141542:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760761.141547:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.141555:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.141569:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760761.178228:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973445 is committed 00002000:00010000:3.0:1789760761.178380:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25100288 -> 25104384 00010000:00010000:3.0:1789760761.178396:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.178413:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f5c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.178422:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f5c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.178429:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760761.178434:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f5c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.178443:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f5c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.178462:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f5c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.178470:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f5c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.178559:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495866240:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37091us (37568us total) trans 4294973445 rc 0/0 00000100:00100000:3.0:1789760761.178569:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6157 00000100:00100000:3.0:1789760761.178579:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745432215716 time 745462167988 state 0x2 ref 0 00000100:00100000:2.0:1789760761.186174:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760761.186325:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495866368 00000100:00100000:1.0:1789760761.186336:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495866368 00000100:00100000:1.0:1789760761.186388:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745432215716 time 745462167988 state 0x0 ref 1 00000100:00100000:1.0:1789760761.186403:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745462428404 time 745462167988 state 0x1 ref 1 00000100:00100000:1.0:1789760761.186436:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745462428404 time 745507152538 state 0x0 ref 1 00000100:00100000:1.0:1789760761.186448:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495866368:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760761.186496:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25104384, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc007 00010000:00010000:1.0:1789760761.186505:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546f63 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.186525:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760761.186530:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.186537:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546f63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.186550:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546f63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760761.218433:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25104384 -> 25108480 00010000:00010000:2.0:1789760761.218461:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546f63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.218474:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546f63 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.218482:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546f63 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.218490:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760761.218495:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546f63 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.218504:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546f63 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.218522:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546f63 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.218529:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354546f63 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.218621:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495866368:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32170us (32453us total) trans 4294973446 rc 0/0 00000100:00100000:2.0:1789760761.218632:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6158 00000100:00100000:2.0:1789760761.218643:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745462428404 time 745507152538 state 0x2 ref 0 00000001:00080000:0.0:1789760761.219415:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973446 is committed 00000100:00100000:1.0:1789760761.225012:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760761.225702:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495866496 00000100:00100000:3.0:1789760761.225713:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495866496 00000100:00100000:3.0:1789760761.225770:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745462428404 time 745507152538 state 0x0 ref 1 00000100:00100000:3.0:1789760761.225786:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745507412954 time 745507152538 state 0x1 ref 1 00000100:00100000:3.0:1789760761.225816:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745507412954 time 745546533292 state 0x0 ref 1 00000100:00100000:3.0:1789760761.225830:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04292a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495866496:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760761.225876:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25108480, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc007 00010000:00010000:3.0:1789760761.225885:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f6a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.225911:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:3.0:1789760761.225923:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.225931:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.225945:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760761.269110:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973447 is committed 00002000:00010000:3.0:1789760761.269353:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25108480 -> 25112576 00010000:00010000:3.0:1789760761.269369:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.269381:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f6a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.269395:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.269402:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760761.269407:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.269415:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f6a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.269434:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f6a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.269441:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f6a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.269709:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04292a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495866496:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43875us (44700us total) trans 4294973447 rc 0/0 00000100:00100000:2.0:1789760761.269720:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6159 00000100:00100000:2.0:1789760761.269731:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745507412954 time 745546533292 state 0x2 ref 0 00000100:00100000:3.0:1789760761.276366:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760761.280656:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495866624 00000100:00100000:2.0:1789760761.280670:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495866624 00000100:00100000:2.0:1789760761.280736:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745507412954 time 745546533292 state 0x0 ref 1 00000100:00100000:2.0:1789760761.280751:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745546793708 time 745546533292 state 0x1 ref 1 00000100:00100000:2.0:1789760761.280790:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745546793708 time 745601507985 state 0x0 ref 1 00000100:00100000:2.0:1789760761.280803:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495866624:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760761.280848:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25112576, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc007 00010000:00010000:2.0:1789760761.280858:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546f71 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.280874:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760761.280878:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.280885:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546f71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.280909:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546f71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760761.317385:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973448 is committed 00002000:00010000:2.0:1789760761.317618:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25112576 -> 25116672 00010000:00010000:2.0:1789760761.317635:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546f71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.317647:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546f71 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.317656:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546f71 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.317664:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760761.317669:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546f71 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.317679:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546f71 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.317698:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546f71 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.317712:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354546f71 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.317941:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495866624:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37066us (41510us total) trans 4294973448 rc 0/0 00000100:00100000:2.0:1789760761.317952:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6160 00000100:00100000:2.0:1789760761.317962:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745546793708 time 745601507985 state 0x2 ref 0 00000100:00100000:3.0:1789760761.322604:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760761.322784:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495866752 00000100:00100000:2.0:1789760761.322794:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495866752 00000100:00100000:2.0:1789760761.322844:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745546793708 time 745601507985 state 0x0 ref 1 00000100:00100000:2.0:1789760761.322858:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745601768401 time 745601507985 state 0x1 ref 1 00000100:00100000:2.0:1789760761.323057:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745601768401 time 745643772345 state 0x0 ref 1 00000100:00100000:2.0:1789760761.323069:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495866752:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760761.323117:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25116672, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc007 00010000:00010000:2.0:1789760761.323125:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f78 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.323142:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760761.323146:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.323152:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.323170:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760761.355898:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25116672 -> 25120768 00010000:00010000:2.0:1789760761.355922:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.355938:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f78 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.355948:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f78 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.355956:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760761.355961:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f78 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.355969:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f78 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.355988:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f78 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.355996:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f78 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.356094:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495866752:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33021us (33492us total) trans 4294973449 rc 0/0 00000100:00100000:2.0:1789760761.356105:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6161 00000100:00100000:2.0:1789760761.356115:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745601768401 time 745643772345 state 0x2 ref 0 00000001:00080000:3.0:1789760761.356466:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973449 is committed 00000100:00100000:1.0:1789760761.363798:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760761.363914:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495866880 00000100:00100000:2.0:1789760761.363925:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495866880 00000100:00100000:2.0:1789760761.363976:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745601768401 time 745643772345 state 0x0 ref 1 00000100:00100000:2.0:1789760761.363991:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745644032761 time 745643772345 state 0x1 ref 1 00000100:00100000:2.0:1789760761.364018:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745644032761 time 745684735142 state 0x0 ref 1 00000100:00100000:2.0:1789760761.364030:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04290700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495866880:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760761.364079:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25120768, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc007 00010000:00010000:2.0:1789760761.364088:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f7f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.364104:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760761.364108:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.364116:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.364130:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760761.401669:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973450 is committed 00002000:00010000:3.0:1789760761.401741:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25120768 -> 25124864 00010000:00010000:3.0:1789760761.401756:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.401769:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f7f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.401783:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f7f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.401791:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760761.401796:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f7f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.401805:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f7f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.401824:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f7f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.401832:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354546f7f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.402220:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04290700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495866880:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38180us (38421us total) trans 4294973450 rc 0/0 00000100:00100000:3.0:1789760761.402231:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6162 00000100:00100000:3.0:1789760761.402241:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745644032761 time 745684735142 state 0x2 ref 0 00000100:00100000:1.0:1789760761.408085:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760761.408245:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495867008 00000100:00100000:2.0:1789760761.408255:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495867008 00000100:00100000:2.0:1789760761.408350:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745644032761 time 745684735142 state 0x0 ref 1 00000100:00100000:2.0:1789760761.408364:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745684995558 time 745684735142 state 0x1 ref 1 00000100:00100000:2.0:1789760761.408391:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745684995558 time 745729107906 state 0x0 ref 1 00000100:00100000:2.0:1789760761.408403:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495867008:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760761.408449:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25124864, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc007 00010000:00010000:2.0:1789760761.408458:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f86 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.408474:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760761.408478:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.408484:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.408497:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760761.447730:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973451 is committed 00002000:00010000:2.0:1789760761.447964:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25124864 -> 25128960 00010000:00010000:2.0:1789760761.448054:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.448075:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f86 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.448084:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f86 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.448092:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760761.448097:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f86 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.448107:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f86 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.448128:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f86 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.448135:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f86 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.448265:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495867008:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39859us (40185us total) trans 4294973451 rc 0/0 00000100:00100000:2.0:1789760761.448276:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6163 00000100:00100000:2.0:1789760761.448339:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745684995558 time 745729107906 state 0x2 ref 0 00000100:00100000:2.0:1789760761.453321:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760761.453547:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495867136 00000100:00100000:1.0:1789760761.453557:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495867136 00000100:00100000:1.0:1789760761.453613:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745684995558 time 745729107906 state 0x0 ref 1 00000100:00100000:1.0:1789760761.453628:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745729368322 time 745729107906 state 0x1 ref 1 00000100:00100000:1.0:1789760761.453664:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745729368322 time 745774380850 state 0x0 ref 1 00000100:00100000:1.0:1789760761.453677:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495867136:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760761.453722:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25128960, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc007 00010000:00010000:1.0:1789760761.453731:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546f8d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.453747:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760761.453752:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.453759:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546f8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.454174:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546f8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760761.493172:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973452 is committed 00002000:00010000:1.0:1789760761.494345:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25128960 -> 25133056 00010000:00010000:1.0:1789760761.494361:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546f8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.494374:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546f8d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.494383:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546f8d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.494390:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760761.494395:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546f8d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.494404:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546f8d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.494423:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546f8d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.494431:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354546f8d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.494525:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495867136:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40846us (41242us total) trans 4294973452 rc 0/0 00000100:00100000:1.0:1789760761.494536:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6164 00000100:00100000:1.0:1789760761.494545:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745729368322 time 745774380850 state 0x2 ref 0 00000100:00100000:1.0:1789760761.499692:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760761.499899:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495867264 00000100:00100000:3.0:1789760761.499909:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495867264 00000100:00100000:3.0:1789760761.499963:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745729368322 time 745774380850 state 0x0 ref 1 00000100:00100000:3.0:1789760761.499980:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745774641266 time 745774380850 state 0x1 ref 1 00000100:00100000:3.0:1789760761.500012:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745774641266 time 745820728650 state 0x0 ref 1 00000100:00100000:3.0:1789760761.500026:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495867264:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760761.500075:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25133056, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc007 00010000:00010000:3.0:1789760761.500085:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f94 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.500102:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:3.0:1789760761.500106:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.500113:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.500128:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760761.535618:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973453 is committed 00002000:00010000:0.0:1789760761.535787:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25133056 -> 25137152 00010000:00010000:0.0:1789760761.535802:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.535813:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f94 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.535822:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f94 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.535829:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760761.535833:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f94 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.535842:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f94 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.535861:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f94 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.535869:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546f94 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760761.535967:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495867264:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35939us (36280us total) trans 4294973453 rc 0/0 00000100:00100000:0.0:1789760761.535976:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6165 00000100:00100000:0.0:1789760761.535986:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745774641266 time 745820728650 state 0x2 ref 0 00000100:00100000:0.0:1789760761.543533:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760761.543680:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495867392 00000100:00100000:2.0:1789760761.543690:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495867392 00000100:00100000:2.0:1789760761.543744:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745774641266 time 745820728650 state 0x0 ref 1 00000100:00100000:2.0:1789760761.543758:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745820989066 time 745820728650 state 0x1 ref 1 00000100:00100000:2.0:1789760761.543788:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745820989066 time 745864504568 state 0x0 ref 1 00000100:00100000:2.0:1789760761.543800:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04929500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495867392:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760761.543847:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25137152, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc007 00010000:00010000:2.0:1789760761.543857:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f9b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.543873:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760761.543877:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.543884:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.543906:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760761.582305:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973454 is committed 00002000:00010000:3.0:1789760761.582458:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25137152 -> 25141248 00010000:00010000:3.0:1789760761.582473:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.582486:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f9b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.582501:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f9b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.582510:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760761.582516:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f9b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.582526:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f9b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.582545:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f9b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.582554:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354546f9b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.582641:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04929500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495867392:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38839us (39123us total) trans 4294973454 rc 0/0 00000100:00100000:3.0:1789760761.582651:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6166 00000100:00100000:3.0:1789760761.582661:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745820989066 time 745864504568 state 0x2 ref 0 00000100:00100000:3.0:1789760761.586350:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760761.586739:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495867520 00000100:00100000:1.0:1789760761.586748:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495867520 00000100:00100000:1.0:1789760761.586803:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745820989066 time 745864504568 state 0x0 ref 1 00000100:00100000:1.0:1789760761.586818:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745864764984 time 745864504568 state 0x1 ref 1 00000100:00100000:1.0:1789760761.586851:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745864764984 time 745907567905 state 0x0 ref 1 00000100:00100000:1.0:1789760761.586863:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495867520:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760761.586921:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25141248, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc007 00010000:00010000:1.0:1789760761.586930:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546fa2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.586946:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760761.586950:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.586957:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546fa2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.586973:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546fa2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760761.622063:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973455 is committed 00002000:00010000:1.0:1789760761.622537:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25141248 -> 25145344 00010000:00010000:1.0:1789760761.622552:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546fa2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.622564:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546fa2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.622573:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546fa2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.622579:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760761.622584:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546fa2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.622592:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546fa2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.622611:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546fa2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.622618:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354546fa2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.622826:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495867520:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35959us (36481us total) trans 4294973455 rc 0/0 00000100:00100000:1.0:1789760761.622839:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6167 00000100:00100000:1.0:1789760761.622849:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745864764984 time 745907567905 state 0x2 ref 0 00000100:00100000:1.0:1789760761.634725:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760761.634871:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495867648 00000100:00100000:0.0:1789760761.634903:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495867648 00000100:00100000:0.0:1789760761.634954:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745864764984 time 745907567905 state 0x0 ref 1 00000100:00100000:0.0:1789760761.634969:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745907828321 time 745907567905 state 0x1 ref 1 00000100:00100000:0.0:1789760761.634999:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745907828321 time 745955715580 state 0x0 ref 1 00000100:00100000:0.0:1789760761.635011:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495867648:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760761.635058:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25145344, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc007 00010000:00010000:0.0:1789760761.635067:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546fa9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.635083:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:0.0:1789760761.635087:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760761.635094:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546fa9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.635107:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546fa9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760761.680198:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25145344 -> 25149440 00002000:00010000:0.0:1789760761.680208:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760760 -> 1789760761 00002000:00010000:0.0:1789760761.680211:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760760 -> 1789760761 00010000:00010000:0.0:1789760761.680223:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546fa9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.680236:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546fa9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.680251:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546fa9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.680260:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760761.680265:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546fa9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.680274:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546fa9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.680316:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546fa9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.680323:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354546fa9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760761.680418:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495867648:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45404us (45698us total) trans 4294973456 rc 0/0 00000100:00100000:0.0:1789760761.680428:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6168 00000100:00100000:0.0:1789760761.680438:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745907828321 time 745955715580 state 0x2 ref 0 00000001:00080000:1.0:1789760761.680629:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973456 is committed 00000100:00100000:0.0:1789760761.688127:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760761.691635:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495867776 00000100:00100000:3.0:1789760761.691650:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495867776 00000100:00100000:3.0:1789760761.691778:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745907828321 time 745955715580 state 0x0 ref 1 00000100:00100000:3.0:1789760761.691799:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745955975996 time 745955715580 state 0x1 ref 1 00000100:00100000:3.0:1789760761.692807:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745955975996 time 746013518894 state 0x0 ref 1 00000100:00100000:3.0:1789760761.692826:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fdca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495867776:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760761.692885:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25149440, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc007 00010000:00010000:3.0:1789760761.692907:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546fb0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.692923:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:3.0:1789760761.692928:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.692934:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546fb0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.692948:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546fb0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.696817:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760761.730481:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25149440 -> 25153536 00010000:00010000:0.0:1789760761.730501:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546fb0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.730520:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546fb0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.730530:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546fb0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.730538:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760761.730543:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546fb0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.730552:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546fb0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.730571:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546fb0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.730579:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354546fb0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760761.730672:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fdca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495867776:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37847us (42549us total) trans 4294973457 rc 0/0 00000100:00100000:0.0:1789760761.730684:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6169 00000100:00100000:0.0:1789760761.730703:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745955975996 time 746013518894 state 0x2 ref 0 00000001:00080000:1.0:1789760761.731223:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973457 is committed 00000100:00100000:2.0:1789760761.739918:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760761.740125:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495867904 00000100:00100000:1.0:1789760761.740134:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495867904 00000100:00100000:1.0:1789760761.740183:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 745955975996 time 746013518894 state 0x0 ref 1 00000100:00100000:1.0:1789760761.740199:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746013779310 time 746013518894 state 0x1 ref 1 00000100:00100000:1.0:1789760761.740237:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746013779310 time 746060953556 state 0x0 ref 1 00000100:00100000:1.0:1789760761.740249:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495867904:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760761.740338:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25153536, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc007 00010000:00010000:1.0:1789760761.740347:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546fb7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.740363:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760761.740367:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.740373:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546fb7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.740387:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546fb7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760761.778480:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973458 is committed 00002000:00010000:1.0:1789760761.778591:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25153536 -> 25157632 00010000:00010000:1.0:1789760761.778685:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546fb7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.778700:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546fb7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.778710:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546fb7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.778717:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760761.778722:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546fb7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.778731:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546fb7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.778751:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546fb7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.778760:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354546fb7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.778910:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495867904:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38656us (38995us total) trans 4294973458 rc 0/0 00000100:00100000:1.0:1789760761.778924:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6170 00000100:00100000:1.0:1789760761.778935:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746013779310 time 746060953556 state 0x2 ref 0 00000100:00100000:2.0:1789760761.795215:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760761.795435:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495868032 00000100:00100000:0.0:1789760761.795446:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495868032 00000100:00100000:0.0:1789760761.795555:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746013779310 time 746060953556 state 0x0 ref 1 00000100:00100000:0.0:1789760761.795572:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746061213972 time 746060953556 state 0x1 ref 1 00000100:00100000:0.0:1789760761.795648:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746061213972 time 746116364193 state 0x0 ref 1 00000100:00100000:0.0:1789760761.795663:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495868032:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760761.795719:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25157632, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc007 00010000:00010000:0.0:1789760761.795729:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546fbe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.795746:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760761.795750:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760761.795757:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546fbe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.795771:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546fbe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760761.836881:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25157632 -> 25161728 00010000:00010000:0.0:1789760761.836909:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546fbe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.836923:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546fbe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.836938:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546fbe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.836945:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760761.836950:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546fbe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.836959:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546fbe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.836976:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546fbe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.836983:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354546fbe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760761.837079:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495868032:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41415us (41868us total) trans 4294973459 rc 0/0 00000100:00100000:0.0:1789760761.837089:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6171 00000100:00100000:0.0:1789760761.837098:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746061213972 time 746116364193 state 0x2 ref 0 00000001:00080000:0.0:1789760761.837134:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973459 is committed 00000100:00100000:0.0:1789760761.842332:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760761.842386:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495868160 00000100:00100000:3.0:1789760761.842398:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495868160 00000100:00100000:3.0:1789760761.842453:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746061213972 time 746116364193 state 0x0 ref 1 00000100:00100000:3.0:1789760761.842467:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746116624609 time 746116364193 state 0x1 ref 1 00000100:00100000:3.0:1789760761.842488:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746116624609 time 746163205627 state 0x0 ref 1 00000100:00100000:3.0:1789760761.842501:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036e57100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495868160:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760761.842546:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25161728, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc007 00010000:00010000:3.0:1789760761.842556:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546fc5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.842572:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:3.0:1789760761.842577:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.842583:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546fc5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.842598:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546fc5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760761.888846:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973460 is committed 00002000:00010000:0.0:1789760761.889001:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25161728 -> 25165824 00010000:00010000:0.0:1789760761.889027:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546fc5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.889041:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546fc5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.889051:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546fc5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.889057:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760761.889062:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546fc5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.889071:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546fc5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.889090:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546fc5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.889098:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354546fc5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760761.889192:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036e57100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495868160:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46688us (46921us total) trans 4294973460 rc 0/0 00000100:00100000:0.0:1789760761.889210:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6172 00000100:00100000:0.0:1789760761.889221:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746116624609 time 746163205627 state 0x2 ref 0 00000100:00100000:1.0:1789760761.899590:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760761.899694:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495868288 00000100:00100000:2.0:1789760761.899703:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495868288 00000100:00100000:2.0:1789760761.899757:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746116624609 time 746163205627 state 0x0 ref 1 00000100:00100000:2.0:1789760761.899772:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746163466043 time 746163205627 state 0x1 ref 1 00000100:00100000:2.0:1789760761.899802:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746163466043 time 746220519855 state 0x0 ref 1 00000100:00100000:2.0:1789760761.899813:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495868288:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760761.899858:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25165824, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc007 00010000:00010000:2.0:1789760761.899879:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546fcc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.899904:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760761.899908:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.899914:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546fcc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.899927:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546fcc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760761.949056:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973461 is committed 00002000:00010000:3.0:1789760761.949207:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25165824 -> 25169920 00002000:00010000:3.0:1789760761.949213:0:6712:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 49159 -> 51209 00010000:00010000:3.0:1789760761.949225:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546fcc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.949237:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546fcc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.949246:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546fcc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.949254:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760761.949259:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546fcc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.949306:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546fcc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.949327:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546fcc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.949336:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354546fcc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.949416:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495868288:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49600us (49831us total) trans 4294973461 rc 0/0 00000100:00100000:3.0:1789760761.949427:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6173 00000100:00100000:3.0:1789760761.949437:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746163466043 time 746220519855 state 0x2 ref 0 00000100:00100000:3.0:1789760761.953004:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760761.953152:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495868416 00000100:00100000:0.0:1789760761.953162:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495868416 00000100:00100000:0.0:1789760761.953219:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746163466043 time 746220519855 state 0x0 ref 1 00000100:00100000:0.0:1789760761.953235:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746220780271 time 746220519855 state 0x1 ref 1 00000100:00100000:0.0:1789760761.953274:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746220780271 time 746273982186 state 0x0 ref 1 00000100:00100000:0.0:1789760761.953314:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495868416:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760761.953362:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25169920, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc809 00010000:00010000:0.0:1789760761.953372:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546fd3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.953388:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760761.953392:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760761.953399:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546fd3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.953413:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546fd3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760761.975665:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973462 is committed 00002000:00010000:1.0:1789760761.975711:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25169920 -> 25174016 00010000:00010000:1.0:1789760761.975726:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546fd3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.975739:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546fd3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.975749:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546fd3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.975757:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ae00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760761.975761:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546fd3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.975771:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546fd3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.975791:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546fd3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.975799:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354546fd3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.975995:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495868416:12345-192.168.203.9@tcp:4:dd.0 Request processed in 22678us (22994us total) trans 4294973462 rc 0/0 00000100:00100000:1.0:1789760761.976010:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6174 00000100:00100000:1.0:1789760761.976080:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746220780271 time 746273982186 state 0x2 ref 0 00000100:00100000:0.0:1789760761.980371:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760761.980562:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495868544 00000100:00100000:2.0:1789760761.980572:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495868544 00000100:00100000:2.0:1789760761.980623:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746220780271 time 746273982186 state 0x0 ref 1 00000100:00100000:2.0:1789760761.980638:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746274242602 time 746273982186 state 0x1 ref 1 00000100:00100000:2.0:1789760761.980666:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746274242602 time 746301381845 state 0x0 ref 1 00000100:00100000:2.0:1789760761.980678:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036e55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495868544:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760761.980728:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25174016, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc809 00010000:00010000:2.0:1789760761.980738:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546fda lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.980753:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760761.980757:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.980764:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546fda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.980786:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546fda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760762.013353:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973463 is committed 00002000:00010000:1.0:1789760762.013531:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25174016 -> 25178112 00010000:00010000:1.0:1789760762.013547:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546fda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.013560:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546fda lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.013568:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546fda lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.013576:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760762.013581:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546fda lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.013591:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546fda lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.013611:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546fda lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.013626:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354546fda lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.013712:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036e55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495868544:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33032us (33351us total) trans 4294973463 rc 0/0 00000100:00100000:1.0:1789760762.013723:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6175 00000100:00100000:1.0:1789760762.013733:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746274242602 time 746301381845 state 0x2 ref 0 00000100:00100000:2.0:1789760762.023121:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760762.023223:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495868672 00000100:00100000:3.0:1789760762.023234:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495868672 00000100:00100000:3.0:1789760762.023323:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746274242602 time 746301381845 state 0x0 ref 1 00000100:00100000:3.0:1789760762.023338:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746301642261 time 746301381845 state 0x1 ref 1 00000100:00100000:3.0:1789760762.023423:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746301642261 time 746344139218 state 0x0 ref 1 00000100:00100000:3.0:1789760762.023436:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495868672:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760762.023484:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25178112, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc809 00010000:00010000:3.0:1789760762.023495:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546fe1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.023512:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:3.0:1789760762.023516:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.023522:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546fe1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.023536:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546fe1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760762.060042:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973464 is committed 00002000:00010000:1.0:1789760762.060196:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25178112 -> 25182208 00010000:00010000:1.0:1789760762.060212:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546fe1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.060225:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546fe1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.060233:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546fe1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.060240:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760762.060246:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546fe1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.060256:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546fe1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.060306:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546fe1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.060314:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354546fe1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.060520:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495868672:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37080us (37402us total) trans 4294973464 rc 0/0 00000100:00100000:1.0:1789760762.060531:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6176 00000100:00100000:1.0:1789760762.060549:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746301642261 time 746344139218 state 0x2 ref 0 00000100:00100000:2.0:1789760762.064684:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760762.064976:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495868800 00000100:00100000:1.0:1789760762.064984:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495868800 00000100:00100000:1.0:1789760762.065033:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746301642261 time 746344139218 state 0x0 ref 1 00000100:00100000:1.0:1789760762.065047:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746344399634 time 746344139218 state 0x1 ref 1 00000100:00100000:1.0:1789760762.065107:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746344399634 time 746385823018 state 0x0 ref 1 00000100:00100000:1.0:1789760762.065118:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495868800:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760762.065162:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25182208, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc809 00010000:00010000:1.0:1789760762.065171:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546fe8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.065185:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760762.065190:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.065195:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546fe8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.065270:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546fe8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.099101:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973465 is committed 00002000:00010000:1.0:1789760762.099343:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25182208 -> 25186304 00010000:00010000:1.0:1789760762.099359:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546fe8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.099371:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546fe8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.099380:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546fe8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.099388:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760762.099393:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546fe8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.099402:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546fe8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.099420:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546fe8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.099432:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546fe8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.099523:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495868800:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34400us (34841us total) trans 4294973465 rc 0/0 00000100:00100000:1.0:1789760762.099533:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6177 00000100:00100000:1.0:1789760762.099544:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746344399634 time 746385823018 state 0x2 ref 0 00000100:00100000:1.0:1789760762.103435:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760762.103517:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495868928 00000100:00100000:2.0:1789760762.103527:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495868928 00000100:00100000:2.0:1789760762.103576:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746344399634 time 746385823018 state 0x0 ref 1 00000100:00100000:2.0:1789760762.103591:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746386083434 time 746385823018 state 0x1 ref 1 00000100:00100000:2.0:1789760762.103641:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746386083434 time 746424357788 state 0x0 ref 1 00000100:00100000:2.0:1789760762.103653:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495868928:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760762.103694:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25186304, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc809 00010000:00010000:2.0:1789760762.103705:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546fef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.103721:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760762.103725:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.103731:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546fef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.103743:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546fef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760762.126962:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973466 is committed 00002000:00010000:2.0:1789760762.127180:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25186304 -> 25190400 00010000:00010000:2.0:1789760762.127196:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546fef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.127619:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546fef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.127631:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546fef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.127639:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760762.127644:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546fef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.127654:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546fef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.127681:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546fef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.127689:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354546fef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.127858:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495868928:12345-192.168.203.9@tcp:4:dd.0 Request processed in 24202us (24426us total) trans 4294973466 rc 0/0 00000100:00100000:2.0:1789760762.127878:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6178 00000100:00100000:2.0:1789760762.127895:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746386083434 time 746424357788 state 0x2 ref 0 00000100:00100000:2.0:1789760762.135803:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760762.136030:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495869056 00000100:00100000:1.0:1789760762.136039:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495869056 00000100:00100000:1.0:1789760762.136101:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746386083434 time 746424357788 state 0x0 ref 1 00000100:00100000:1.0:1789760762.136116:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746424618204 time 746424357788 state 0x1 ref 1 00000100:00100000:1.0:1789760762.136148:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746424618204 time 746456864892 state 0x0 ref 1 00000100:00100000:1.0:1789760762.136161:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495869056:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760762.136208:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25190400, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc809 00010000:00010000:1.0:1789760762.136217:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546ff6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.136232:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760762.136237:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.136242:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546ff6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.136256:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546ff6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760762.177076:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973467 is committed 00002000:00010000:0.0:1789760762.177150:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25190400 -> 25194496 00010000:00010000:0.0:1789760762.177165:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546ff6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.177185:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546ff6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.177194:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546ff6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.177200:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760762.177205:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546ff6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.178064:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546ff6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.178088:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546ff6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.178096:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354546ff6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.178225:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495869056:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42062us (42428us total) trans 4294973467 rc 0/0 00000100:00100000:0.0:1789760762.178234:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6179 00000100:00100000:0.0:1789760762.178244:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746424618204 time 746456864892 state 0x2 ref 0 00000100:00100000:3.0:1789760762.182855:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760762.183437:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495869184 00000100:00100000:1.0:1789760762.183448:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495869184 00000100:00100000:1.0:1789760762.183502:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746424618204 time 746456864892 state 0x0 ref 1 00000100:00100000:1.0:1789760762.183516:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746457125308 time 746456864892 state 0x1 ref 1 00000100:00100000:1.0:1789760762.183548:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746457125308 time 746504264272 state 0x0 ref 1 00000100:00100000:1.0:1789760762.183560:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495869184:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760762.183609:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25194496, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc809 00010000:00010000:1.0:1789760762.183618:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546ffd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.183634:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760762.183638:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.183644:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546ffd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.183658:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546ffd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760762.231478:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973468 is committed 00002000:00010000:2.0:1789760762.231627:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25194496 -> 25198592 00010000:00010000:2.0:1789760762.231642:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546ffd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.231654:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546ffd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.231661:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546ffd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.231665:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760762.231668:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546ffd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.231676:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546ffd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.231690:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546ffd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.231695:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354546ffd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.231858:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495869184:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48295us (49008us total) trans 4294973468 rc 0/0 00000100:00100000:2.0:1789760762.231869:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6180 00000100:00100000:2.0:1789760762.231879:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746457125308 time 746504264272 state 0x2 ref 0 00000100:00100000:1.0:1789760762.236221:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760762.238081:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495869312 00000100:00100000:2.0:1789760762.238098:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495869312 00000100:00100000:2.0:1789760762.238566:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746457125308 time 746504264272 state 0x0 ref 1 00000100:00100000:2.0:1789760762.238588:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746504524688 time 746504264272 state 0x1 ref 1 00000100:00100000:3.0:1789760762.238707:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746504524688 time 746559419805 state 0x0 ref 1 00000100:00100000:3.0:1789760762.238723:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903639ac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495869312:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760762.238773:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25198592, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc809 00010000:00010000:3.0:1789760762.238783:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547004 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.238798:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:3.0:1789760762.238815:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.238822:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547004 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.238835:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547004 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760762.290426:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973469 is committed 00002000:00010000:1.0:1789760762.290547:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25198592 -> 25202688 00010000:00010000:1.0:1789760762.290562:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547004 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.290575:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547004 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.290584:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547004 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.290592:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760762.290597:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547004 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.290606:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547004 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.290625:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547004 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.290632:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547004 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.290733:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903639ac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495869312:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52010us (54518us total) trans 4294973469 rc 0/0 00000100:00100000:1.0:1789760762.290744:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6181 00000100:00100000:1.0:1789760762.290753:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746504524688 time 746559419805 state 0x2 ref 0 00000100:00100000:1.0:1789760762.305499:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760762.305603:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495869440 00000100:00100000:0.0:1789760762.305614:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495869440 00000100:00100000:0.0:1789760762.305667:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746504524688 time 746559419805 state 0x0 ref 1 00000100:00100000:0.0:1789760762.305681:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746559680221 time 746559419805 state 0x1 ref 1 00000100:00100000:0.0:1789760762.305712:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746559680221 time 746626429164 state 0x0 ref 1 00000100:00100000:0.0:1789760762.305724:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495869440:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760762.305771:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25202688, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc809 00010000:00010000:0.0:1789760762.305780:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454700b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.305796:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760762.305800:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.305807:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454700b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.305821:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454700b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760762.332902:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973470 is committed 00002000:00010000:1.0:1789760762.333414:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25202688 -> 25206784 00010000:00010000:1.0:1789760762.333428:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454700b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.333441:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454700b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.333451:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454700b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.333458:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ae00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760762.333463:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454700b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.333473:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454700b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.333492:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454700b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.333499:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454700b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.333585:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495869440:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27859us (28090us total) trans 4294973470 rc 0/0 00000100:00100000:1.0:1789760762.333600:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6182 00000100:00100000:1.0:1789760762.333609:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746559680221 time 746626429164 state 0x2 ref 0 00000100:00100000:0.0:1789760762.345605:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760762.345936:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495869568 00000100:00100000:2.0:1789760762.345945:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495869568 00000100:00100000:2.0:1789760762.345995:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746559680221 time 746626429164 state 0x0 ref 1 00000100:00100000:2.0:1789760762.346010:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746626689580 time 746626429164 state 0x1 ref 1 00000100:00100000:2.0:1789760762.346081:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746626689580 time 746666797740 state 0x0 ref 1 00000100:00100000:2.0:1789760762.346095:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036e57100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495869568:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760762.346142:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25206784, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc809 00010000:00010000:2.0:1789760762.346154:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547012 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.346173:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760762.346177:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.346182:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547012 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.346195:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547012 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760762.380445:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25206784 -> 25210880 00010000:00010000:0.0:1789760762.380465:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547012 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.380485:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547012 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.380494:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547012 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.380501:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760762.380506:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547012 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.380516:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547012 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.380535:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547012 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.380544:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547012 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.380635:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036e57100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495869568:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34537us (35035us total) trans 4294973471 rc 0/0 00000100:00100000:0.0:1789760762.380646:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6183 00000100:00100000:0.0:1789760762.380656:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746626689580 time 746666797740 state 0x2 ref 0 00000001:00080000:0.0:1789760762.380697:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973471 is committed 00000100:00100000:0.0:1789760762.392792:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760762.392907:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495869696 00000100:00100000:3.0:1789760762.392916:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495869696 00000100:00100000:3.0:1789760762.392971:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746626689580 time 746666797740 state 0x0 ref 1 00000100:00100000:3.0:1789760762.392986:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746667058156 time 746666797740 state 0x1 ref 1 00000100:00100000:3.0:1789760762.393019:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746667058156 time 746713736096 state 0x0 ref 1 00000100:00100000:3.0:1789760762.393031:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036e55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495869696:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760762.393082:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25210880, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc809 00010000:00010000:3.0:1789760762.393094:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547019 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.393113:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:3.0:1789760762.393118:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.393125:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547019 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.393142:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547019 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760762.429222:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25210880 -> 25214976 00010000:00010000:3.0:1789760762.429241:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547019 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.429270:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547019 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.429362:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547019 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.429371:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760762.429375:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547019 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.429384:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547019 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.429404:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547019 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.429411:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547019 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.429503:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036e55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495869696:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36470us (36717us total) trans 4294973472 rc 0/0 00000100:00100000:3.0:1789760762.429513:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6184 00000100:00100000:3.0:1789760762.429524:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746667058156 time 746713736096 state 0x2 ref 0 00000001:00080000:2.0:1789760762.429597:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973472 is committed 00000100:00100000:2.0:1789760762.435069:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760762.435480:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495869824 00000100:00100000:1.0:1789760762.435488:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495869824 00000100:00100000:1.0:1789760762.435542:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746667058156 time 746713736096 state 0x0 ref 1 00000100:00100000:1.0:1789760762.435558:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746713996512 time 746713736096 state 0x1 ref 1 00000100:00100000:1.0:1789760762.435634:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746713996512 time 746756350559 state 0x0 ref 1 00000100:00100000:1.0:1789760762.435645:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034ce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495869824:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760762.435692:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25214976, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc809 00010000:00010000:1.0:1789760762.435700:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547020 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.435716:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760762.435720:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.435726:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547020 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.435739:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547020 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760762.473528:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25214976 -> 25219072 00010000:00010000:1.0:1789760762.473548:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547020 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.473561:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547020 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.473569:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547020 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.473576:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760762.473636:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547020 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.473645:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547020 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.473664:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547020 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.473671:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547020 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.473751:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034ce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495869824:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38103us (38686us total) trans 4294973473 rc 0/0 00000100:00100000:1.0:1789760762.473987:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6185 00000100:00100000:1.0:1789760762.473997:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746713996512 time 746756350559 state 0x2 ref 0 00000001:00080000:2.0:1789760762.474134:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973473 is committed 00000100:00100000:0.0:1789760762.485042:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760762.485147:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495869952 00000100:00100000:2.0:1789760762.485157:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495869952 00000100:00100000:2.0:1789760762.485210:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746713996512 time 746756350559 state 0x0 ref 1 00000100:00100000:2.0:1789760762.485225:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746756610975 time 746756350559 state 0x1 ref 1 00000100:00100000:2.0:1789760762.485257:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746756610975 time 746805973929 state 0x0 ref 1 00000100:00100000:2.0:1789760762.485269:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929018a80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495869952:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760762.485348:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25219072, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc809 00010000:00010000:2.0:1789760762.485358:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547027 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.485373:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760762.485377:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.485383:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547027 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.485396:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547027 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.523962:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973474 is committed 00002000:00010000:0.0:1789760762.524036:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25219072 -> 25223168 00010000:00010000:0.0:1789760762.524049:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547027 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.524059:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547027 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.524065:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547027 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.524070:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760762.524073:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547027 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.524081:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547027 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.524096:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547027 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.524100:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547027 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.524176:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929018a80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495869952:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38904us (39138us total) trans 4294973474 rc 0/0 00000100:00100000:0.0:1789760762.524189:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6186 00000100:00100000:0.0:1789760762.524199:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746756610975 time 746805973929 state 0x2 ref 0 00000100:00100000:0.0:1789760762.530445:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760762.530515:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495870080 00000100:00100000:3.0:1789760762.530525:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495870080 00000100:00100000:3.0:1789760762.530585:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746756610975 time 746805973929 state 0x0 ref 1 00000100:00100000:3.0:1789760762.530602:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746806234345 time 746805973929 state 0x1 ref 1 00000100:00100000:3.0:1789760762.530632:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746806234345 time 746851349146 state 0x0 ref 1 00000100:00100000:3.0:1789760762.530645:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495870080:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760762.530690:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25223168, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc809 00010000:00010000:3.0:1789760762.530699:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454702e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.530714:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:3.0:1789760762.530718:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.530725:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454702e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.530739:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454702e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760762.569094:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973475 is committed 00002000:00010000:0.0:1789760762.569180:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25223168 -> 25227264 00010000:00010000:0.0:1789760762.569194:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454702e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.569207:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454702e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.569216:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454702e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.569223:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760762.569228:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454702e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.569237:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454702e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.569256:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454702e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.569263:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454702e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.569911:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495870080:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39259us (39466us total) trans 4294973475 rc 0/0 00000100:00100000:0.0:1789760762.569922:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6187 00000100:00100000:0.0:1789760762.569932:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746806234345 time 746851349146 state 0x2 ref 0 00000100:00100000:2.0:1789760762.573803:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760762.573922:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495870208 00000100:00100000:1.0:1789760762.573932:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495870208 00000100:00100000:1.0:1789760762.573986:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746806234345 time 746851349146 state 0x0 ref 1 00000100:00100000:1.0:1789760762.574003:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746851609562 time 746851349146 state 0x1 ref 1 00000100:00100000:1.0:1789760762.574029:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746851609562 time 746894746096 state 0x0 ref 1 00000100:00100000:1.0:1789760762.574042:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495870208:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760762.574094:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25227264, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc809 00010000:00010000:1.0:1789760762.574104:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547035 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.574120:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760762.574125:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.574132:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547035 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.574146:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547035 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.617059:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973476 is committed 00002000:00010000:1.0:1789760762.617161:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25227264 -> 25231360 00010000:00010000:1.0:1789760762.617178:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547035 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.617190:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547035 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.617277:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547035 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.617317:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760762.617323:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547035 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.617332:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547035 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.617351:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547035 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.617358:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547035 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.617448:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495870208:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43405us (43657us total) trans 4294973476 rc 0/0 00000100:00100000:1.0:1789760762.617458:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6188 00000100:00100000:1.0:1789760762.617466:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746851609562 time 746894746096 state 0x2 ref 0 00000100:00100000:3.0:1789760762.628019:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760762.628409:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495870336 00000100:00100000:0.0:1789760762.628419:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495870336 00000100:00100000:0.0:1789760762.628473:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746851609562 time 746894746096 state 0x0 ref 1 00000100:00100000:0.0:1789760762.628489:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746895006512 time 746894746096 state 0x1 ref 1 00000100:00100000:0.0:1789760762.628522:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746895006512 time 746949239354 state 0x0 ref 1 00000100:00100000:0.0:1789760762.628534:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495870336:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760762.628593:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25231360, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc809 00010000:00010000:0.0:1789760762.628602:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454703c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.628618:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760762.628623:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.628629:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454703c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.628642:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454703c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760762.657039:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973477 is committed 00002000:00010000:2.0:1789760762.657347:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25231360 -> 25235456 00002000:00010000:2.0:1789760762.657352:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760761 -> 1789760762 00002000:00010000:2.0:1789760762.657355:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760761 -> 1789760762 00010000:00010000:2.0:1789760762.657577:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454703c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.657591:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454703c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.657602:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454703c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.657614:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760762.657620:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454703c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.657634:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454703c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.657655:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454703c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.657662:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x493674835454703c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.657848:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495870336:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29310us (29834us total) trans 4294973477 rc 0/0 00000100:00100000:2.0:1789760762.657859:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6189 00000100:00100000:2.0:1789760762.657876:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746895006512 time 746949239354 state 0x2 ref 0 00000100:00100000:3.0:1789760762.665388:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760762.665854:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495870464 00000100:00100000:0.0:1789760762.665865:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495870464 00000100:00100000:0.0:1789760762.665988:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746895006512 time 746949239354 state 0x0 ref 1 00000100:00100000:0.0:1789760762.666008:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746949499770 time 746949239354 state 0x1 ref 1 00000100:00100000:0.0:1789760762.666034:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746949499770 time 746986750873 state 0x0 ref 1 00000100:00100000:0.0:1789760762.666046:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495870464:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760762.666096:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25235456, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc809 00010000:00010000:0.0:1789760762.666105:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547043 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.666121:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760762.666125:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.666132:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547043 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.666144:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547043 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760762.708778:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973478 is committed 00002000:00010000:2.0:1789760762.709022:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25235456 -> 25239552 00010000:00010000:2.0:1789760762.709038:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547043 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.709051:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547043 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.709061:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547043 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.709068:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760762.709074:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547043 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.709083:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547043 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.709114:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547043 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.709123:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547043 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.709218:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495870464:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43169us (43834us total) trans 4294973478 rc 0/0 00000100:00100000:2.0:1789760762.709236:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6190 00000100:00100000:2.0:1789760762.709246:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746949499770 time 746986750873 state 0x2 ref 0 00000100:00100000:1.0:1789760762.715500:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760762.715603:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495870592 00000100:00100000:2.0:1789760762.715613:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495870592 00000100:00100000:2.0:1789760762.715667:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746949499770 time 746986750873 state 0x0 ref 1 00000100:00100000:2.0:1789760762.715684:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746987011289 time 746986750873 state 0x1 ref 1 00000100:00100000:2.0:1789760762.715719:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746987011289 time 747036435505 state 0x0 ref 1 00000100:00100000:2.0:1789760762.715730:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495870592:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760762.715776:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25239552, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc809 00010000:00010000:2.0:1789760762.715785:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454704a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.715800:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760762.715805:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.715812:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454704a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.715825:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454704a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.719303:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760762.753945:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973479 is committed 00002000:00010000:2.0:1789760762.754031:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25239552 -> 25243648 00010000:00010000:2.0:1789760762.754048:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454704a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.754061:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454704a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.754071:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454704a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.754078:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760762.754084:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454704a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.754093:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454704a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.754118:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454704a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.754126:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454704a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.754370:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495870592:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38636us (38874us total) trans 4294973479 rc 0/0 00000100:00100000:2.0:1789760762.754382:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6191 00000100:00100000:2.0:1789760762.754392:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746987011289 time 747036435505 state 0x2 ref 0 00000100:00100000:3.0:1789760762.758082:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760762.758381:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495870720 00000100:00100000:2.0:1789760762.758391:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495870720 00000100:00100000:2.0:1789760762.758442:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 746987011289 time 747036435505 state 0x0 ref 1 00000100:00100000:2.0:1789760762.758457:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747036695921 time 747036435505 state 0x1 ref 1 00000100:00100000:2.0:1789760762.758480:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747036695921 time 747079197909 state 0x0 ref 1 00000100:00100000:2.0:1789760762.758493:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495870720:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760762.758536:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25243648, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc809 00010000:00010000:2.0:1789760762.758545:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547051 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.758561:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760762.758566:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.758572:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547051 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.758585:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547051 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.783708:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973480 is committed 00002000:00010000:2.0:1789760762.783776:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25243648 -> 25247744 00010000:00010000:2.0:1789760762.783792:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547051 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.783807:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547051 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.783817:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547051 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.783825:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760762.783835:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547051 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.783845:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547051 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.783864:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547051 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.783893:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547051 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.783985:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495870720:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25489us (25912us total) trans 4294973480 rc 0/0 00000100:00100000:2.0:1789760762.783995:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6192 00000100:00100000:2.0:1789760762.784003:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747036695921 time 747079197909 state 0x2 ref 0 00000100:00100000:3.0:1789760762.787231:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760762.787327:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495870848 00000100:00100000:2.0:1789760762.787337:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495870848 00000100:00100000:2.0:1789760762.787395:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747036695921 time 747079197909 state 0x0 ref 1 00000100:00100000:2.0:1789760762.787409:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747079458325 time 747079197909 state 0x1 ref 1 00000100:00100000:2.0:1789760762.787443:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747079458325 time 747108159876 state 0x0 ref 1 00000100:00100000:2.0:1789760762.787455:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495870848:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760762.787502:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25247744, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc809 00010000:00010000:2.0:1789760762.787511:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547058 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.787526:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760762.787531:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.787538:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547058 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.787557:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547058 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760762.813799:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973481 is committed 00002000:00010000:0.0:1789760762.814114:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25247744 -> 25251840 00010000:00010000:0.0:1789760762.814130:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547058 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.814143:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547058 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.814152:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547058 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.814160:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760762.814165:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547058 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.814174:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547058 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.814193:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547058 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.814201:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547058 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.814323:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495870848:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26864us (27096us total) trans 4294973481 rc 0/0 00000100:00100000:0.0:1789760762.814333:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6193 00000100:00100000:0.0:1789760762.814343:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747079458325 time 747108159876 state 0x2 ref 0 00000100:00100000:1.0:1789760762.817592:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760762.817683:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495870976 00000100:00100000:2.0:1789760762.817692:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495870976 00000100:00100000:2.0:1789760762.817744:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747079458325 time 747108159876 state 0x0 ref 1 00000100:00100000:2.0:1789760762.817761:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747108420292 time 747108159876 state 0x1 ref 1 00000100:00100000:2.0:1789760762.817826:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747108420292 time 747138542293 state 0x0 ref 1 00000100:00100000:2.0:1789760762.817838:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495870976:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760762.817903:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25251840, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc809 00010000:00010000:2.0:1789760762.817912:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454705f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.817928:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760762.817932:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.817939:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454705f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.817961:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454705f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760762.855929:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973482 is committed 00002000:00010000:3.0:1789760762.856353:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25251840 -> 25255936 00010000:00010000:3.0:1789760762.856368:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454705f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.856379:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454705f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.856387:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454705f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.856394:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760762.856398:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454705f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.856406:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454705f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.856424:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454705f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.856432:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454705f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.856563:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495870976:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38723us (38977us total) trans 4294973482 rc 0/0 00000100:00100000:3.0:1789760762.856574:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6194 00000100:00100000:3.0:1789760762.856586:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747108420292 time 747138542293 state 0x2 ref 0 00000100:00100000:3.0:1789760762.873854:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760762.873963:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495871104 00000100:00100000:2.0:1789760762.873973:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495871104 00000100:00100000:2.0:1789760762.874025:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747108420292 time 747138542293 state 0x0 ref 1 00000100:00100000:2.0:1789760762.874040:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747138802709 time 747138542293 state 0x1 ref 1 00000100:00100000:2.0:1789760762.874070:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747138802709 time 747194786761 state 0x0 ref 1 00000100:00100000:2.0:1789760762.874080:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495871104:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760762.874124:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25255936, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc809 00010000:00010000:2.0:1789760762.874132:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547066 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.874147:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760762.874151:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.874158:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547066 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.874170:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547066 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760762.920921:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973483 is committed 00002000:00010000:3.0:1789760762.920964:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25255936 -> 25260032 00010000:00010000:3.0:1789760762.920980:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547066 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.920993:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547066 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.921003:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547066 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.921010:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760762.921015:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547066 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.921025:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547066 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.921045:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547066 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.921051:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547066 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.921161:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495871104:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47075us (47309us total) trans 4294973483 rc 0/0 00000100:00100000:3.0:1789760762.921176:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6195 00000100:00100000:3.0:1789760762.921187:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747138802709 time 747194786761 state 0x2 ref 0 00000100:00100000:0.0:1789760762.924418:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760762.924574:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495871232 00000100:00100000:2.0:1789760762.924585:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495871232 00000100:00100000:2.0:1789760762.924639:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747138802709 time 747194786761 state 0x0 ref 1 00000100:00100000:2.0:1789760762.924654:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747195047177 time 747194786761 state 0x1 ref 1 00000100:00100000:2.0:1789760762.924686:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747195047177 time 747245402303 state 0x0 ref 1 00000100:00100000:2.0:1789760762.924697:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495871232:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760762.924747:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25260032, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc809 00010000:00010000:2.0:1789760762.924756:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454706d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.924771:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760762.924775:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.924781:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454706d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.924795:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454706d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760762.976967:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973484 is committed 00002000:00010000:2.0:1789760762.977113:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25260032 -> 25264128 00010000:00010000:2.0:1789760762.977130:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454706d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.977143:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454706d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.977152:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454706d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.977160:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760762.977165:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454706d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.977175:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454706d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.977194:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454706d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.977202:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454706d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.977502:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495871232:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52801us (53100us total) trans 4294973484 rc 0/0 00000100:00100000:2.0:1789760762.977513:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6196 00000100:00100000:2.0:1789760762.977524:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747195047177 time 747245402303 state 0x2 ref 0 00000100:00100000:1.0:1789760762.983701:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760762.984196:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495871360 00000100:00100000:3.0:1789760762.984207:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495871360 00000100:00100000:3.0:1789760762.984260:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747195047177 time 747245402303 state 0x0 ref 1 00000100:00100000:3.0:1789760762.984323:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747245662719 time 747245402303 state 0x1 ref 1 00000100:00100000:3.0:1789760762.984420:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747245662719 time 747305134727 state 0x0 ref 1 00000100:00100000:3.0:1789760762.984433:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495871360:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760762.984484:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25264128, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc809 00010000:00010000:3.0:1789760762.984493:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547074 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.984510:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:3.0:1789760762.984514:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.984521:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547074 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.984535:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547074 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760763.020601:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25264128 -> 25268224 00010000:00010000:2.0:1789760763.020620:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547074 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.020633:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547074 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.020648:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547074 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.020656:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760763.020661:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547074 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.020671:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547074 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.020691:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547074 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.020699:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547074 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760763.020865:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973485 is committed 00000100:00100000:2.0:1789760763.021082:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495871360:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36646us (37385us total) trans 4294973485 rc 0/0 00000100:00100000:2.0:1789760763.021093:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6197 00000100:00100000:2.0:1789760763.021103:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747245662719 time 747305134727 state 0x2 ref 0 00000100:00100000:0.0:1789760763.027074:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760763.027223:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495871488 00000100:00100000:3.0:1789760763.027234:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495871488 00000100:00080000:2.0:1789760763.027419:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760763.027590:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747245662719 time 747305134727 state 0x0 ref 1 00000100:00100000:3.0:1789760763.027606:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747305395143 time 747305134727 state 0x1 ref 1 00000100:00100000:3.0:1789760763.027683:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747305395143 time 747348400180 state 0x0 ref 1 00000100:00100000:3.0:1789760763.027697:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495871488:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760763.027743:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25268224, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc809 00010000:00010000:3.0:1789760763.027819:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454707b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.027955:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:3.0:1789760763.027961:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.027967:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454707b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.027982:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454707b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760763.065332:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973486 is committed 00002000:00010000:1.0:1789760763.065503:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25268224 -> 25272320 00010000:00010000:1.0:1789760763.065520:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454707b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.065533:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454707b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.065541:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454707b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.065548:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760763.065553:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454707b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.065562:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454707b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.065580:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454707b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.065588:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454707b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.065676:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495871488:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37978us (38607us total) trans 4294973486 rc 0/0 00000100:00100000:1.0:1789760763.065687:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6198 00000100:00100000:1.0:1789760763.065697:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747305395143 time 747348400180 state 0x2 ref 0 00000100:00100000:1.0:1789760763.069696:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760763.069785:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495871616 00000100:00100000:2.0:1789760763.069796:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495871616 00000100:00100000:2.0:1789760763.069862:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747305395143 time 747348400180 state 0x0 ref 1 00000100:00100000:2.0:1789760763.069877:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747348660596 time 747348400180 state 0x1 ref 1 00000100:00100000:2.0:1789760763.069920:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747348660596 time 747390636333 state 0x0 ref 1 00000100:00100000:2.0:1789760763.069932:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495871616:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760763.069981:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25272320, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc809 00010000:00010000:2.0:1789760763.069991:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547082 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.070007:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760763.070012:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760763.070018:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547082 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.070032:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547082 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760763.098610:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973487 is committed 00002000:00010000:2.0:1789760763.098667:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25272320 -> 25276416 00010000:00010000:2.0:1789760763.098682:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547082 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.098695:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547082 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.098704:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547082 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.098713:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760763.098719:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547082 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.098728:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547082 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.098747:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547082 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.098755:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547082 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760763.098868:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495871616:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28925us (29168us total) trans 4294973487 rc 0/0 00000100:00100000:2.0:1789760763.098888:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6199 00000100:00100000:2.0:1789760763.098899:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747348660596 time 747390636333 state 0x2 ref 0 00000100:00100000:2.0:1789760763.107425:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760763.107542:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495871744 00000100:00100000:3.0:1789760763.107558:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495871744 00000100:00100000:3.0:1789760763.107637:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747348660596 time 747390636333 state 0x0 ref 1 00000100:00100000:3.0:1789760763.107655:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747390896749 time 747390636333 state 0x1 ref 1 00000100:00100000:3.0:1789760763.107690:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747390896749 time 747428406103 state 0x0 ref 1 00000100:00100000:3.0:1789760763.107705:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495871744:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760763.107771:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25276416, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc809 00010000:00010000:3.0:1789760763.107782:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547089 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.107802:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:3.0:1789760763.107807:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.107814:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547089 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.107830:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547089 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760763.153220:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973488 is committed 00002000:00010000:0.0:1789760763.153341:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25276416 -> 25280512 00010000:00010000:0.0:1789760763.153356:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547089 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.153369:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547089 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.153379:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547089 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.153386:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760763.153391:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547089 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.153401:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547089 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.153419:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547089 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.153426:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547089 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.153603:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495871744:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45897us (46182us total) trans 4294973488 rc 0/0 00000100:00100000:0.0:1789760763.153613:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6200 00000100:00100000:0.0:1789760763.153623:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747390896749 time 747428406103 state 0x2 ref 0 00000100:00100000:3.0:1789760763.156853:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760763.157044:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495871872 00000100:00100000:1.0:1789760763.157055:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495871872 00000100:00100000:1.0:1789760763.157115:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747390896749 time 747428406103 state 0x0 ref 1 00000100:00100000:1.0:1789760763.157134:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747428666519 time 747428406103 state 0x1 ref 1 00000100:00100000:1.0:1789760763.157166:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747428666519 time 747477882312 state 0x0 ref 1 00000100:00100000:1.0:1789760763.157180:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495871872:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760763.157240:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25280512, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc809 00010000:00010000:1.0:1789760763.157251:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547090 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.157270:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760763.157276:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.157316:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547090 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.157327:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547090 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.171042:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f034cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:59c4e264-9b75-482b-8813-3d1dc124f384:3317:1876699520311168:0@lo:400:kworker.0 00000100:00100000:2.0:1789760763.171072:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f034cc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3318:1876699520311296:0@lo:400:kworker.0 00000100:00100000:1.0:1789760763.171106:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760763.171366:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520311168 00000100:00100000:0.0:1789760763.171375:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520311168 00000100:00100000:0.0:1789760763.171419:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 409 00000100:00100000:0.0:1789760763.171434:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929012313450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:59c4e264-9b75-482b-8813-3d1dc124f384+19:3317:x1876699520311168:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760763.172562:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760763.172632:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f034ce300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3318:1876699520311808:0@lo:400:kworker.0 00000100:00100000:2.0:1789760763.172678:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760763.172705:0:3318:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760763.172712:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f034cd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3318:1876699520311680:0@lo:400:kworker.0 00000100:00100000:2.0:1789760763.172736:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760763.172771:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f034cdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520311424:0@lo:400:kworker.0 00000100:00100000:2.0:1789760763.172791:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760763.172808:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789760763.172813:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f034cca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520311552:0@lo:400:kworker.0 00000100:00100000:2.0:1789760763.172832:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760763.173001:0:12710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520311296 00000100:00100000:2.0:1789760763.173009:0:12710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520311296 00000100:00100000:2.0:1789760763.173096:0:12710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520311680 00000100:00100000:2.0:1789760763.173100:0:12710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520311680 00000100:00100000:1.0:1789760763.173155:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520311424 00000100:00100000:1.0:1789760763.173159:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520311424 00000100:00100000:1.0:1789760763.173187:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 502 00000100:00100000:1.0:1789760763.173194:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3318:x1876699520311296:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760763.173257:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3318:x1876699520311296:12345-0@lo:400:kworker.0 Request processed in 63us (697us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760763.173264:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 502 00000100:00100000:1.0:1789760763.173296:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 503 00000100:00100000:1.0:1789760763.173302:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1876699520311680:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760763.173328:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1876699520311680:12345-0@lo:400:kworker.0 Request processed in 27us (592us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760763.173335:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 503 00000100:00100000:1.0:1789760763.173345:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 504 00000100:00100000:1.0:1789760763.173364:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520311424:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760763.173397:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520311424:12345-0@lo:400:kworker.0 Request processed in 46us (604us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760763.173402:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 504 00000100:00100000:1.0:1789760763.173438:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f034cdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520311424:0@lo:400:kworker.0 00000100:00100000:1.0:1789760763.173464:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f034cd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3318:1876699520311680:0@lo:400:kworker.0 00000100:00100000:1.0:1789760763.173473:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f034cc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3318:1876699520311296:0@lo:400:kworker.0 00000100:00100000:2.0:1789760763.173819:0:13058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520311808 00000100:00100000:2.0:1789760763.173823:0:13058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520311808 00000100:00100000:2.0:1789760763.173848:0:13058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520311552 00000100:00100000:2.0:1789760763.173851:0:13058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520311552 00000100:00100000:0.0:1789760763.173903:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929012313450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:59c4e264-9b75-482b-8813-3d1dc124f384+19:3317:x1876699520311168:12345-0@lo:400:kworker.0 Request processed in 2471us (2795us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760763.173911:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 409 00000100:00100000:2.0:1789760763.173927:0:13058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 232 00000100:00100000:2.0:1789760763.173936:0:13058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3318:x1876699520311808:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760763.173974:0:13058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3318:x1876699520311808:12345-0@lo:400:kworker.0 Request processed in 39us (1294us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760763.173980:0:13058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 232 00000100:00100000:2.0:1789760763.173991:0:13058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 233 00000100:00100000:2.0:1789760763.173997:0:13058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520311552:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760763.174037:0:13058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520311552:12345-0@lo:400:kworker.0 Request processed in 40us (1203us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760763.174043:0:13058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 233 00000100:00100000:1.0:1789760763.174087:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f034cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:59c4e264-9b75-482b-8813-3d1dc124f384:3317:1876699520311168:0@lo:400:kworker.0 00000100:00100000:0.0:1789760763.174089:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f034cca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520311552:0@lo:400:kworker.0 00000100:00100000:1.0:1789760763.174112:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f034ce300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3318:1876699520311808:0@lo:400:kworker.0 00000100:00080000:2.0:1789760763.175186:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789760763.175192:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760763.175195:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:3.0:1789760763.210712:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973489 is committed 00002000:00010000:0.0:1789760763.210824:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25280512 -> 25284608 00010000:00010000:0.0:1789760763.210840:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547090 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.210853:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547090 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.210874:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547090 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.210890:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760763.210897:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547090 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.210905:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547090 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.210923:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547090 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.210930:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547090 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.211020:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495871872:12345-192.168.203.9@tcp:4:dd.0 Request processed in 53839us (54171us total) trans 4294973489 rc 0/0 00000100:00100000:0.0:1789760763.211031:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6201 00000100:00100000:0.0:1789760763.211042:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747428666519 time 747477882312 state 0x2 ref 0 00000100:00100000:1.0:1789760763.215403:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760763.215569:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495872000 00000100:00100000:2.0:1789760763.215580:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495872000 00000100:00100000:2.0:1789760763.215634:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747428666519 time 747477882312 state 0x0 ref 1 00000100:00100000:2.0:1789760763.215649:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747478142728 time 747477882312 state 0x1 ref 1 00000100:00100000:2.0:1789760763.215679:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747478142728 time 747536395568 state 0x0 ref 1 00000100:00100000:2.0:1789760763.215784:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495872000:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760763.215837:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25284608, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc809 00010000:00010000:2.0:1789760763.215847:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547097 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.215945:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760763.215951:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760763.215957:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547097 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.215972:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547097 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760763.246727:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973490 is committed 00002000:00010000:2.0:1789760763.246822:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25284608 -> 25288704 00010000:00010000:2.0:1789760763.246838:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547097 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.246850:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547097 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.246875:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547097 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.246892:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760763.246897:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547097 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.246906:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547097 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.246926:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547097 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.246933:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547097 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760763.247016:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495872000:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31323us (31620us total) trans 4294973490 rc 0/0 00000100:00100000:2.0:1789760763.247025:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6202 00000100:00100000:2.0:1789760763.247035:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747478142728 time 747536395568 state 0x2 ref 0 00000100:00100000:3.0:1789760763.251315:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760763.251713:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495872128 00000100:00100000:2.0:1789760763.251724:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495872128 00000100:00100000:2.0:1789760763.251780:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747478142728 time 747536395568 state 0x0 ref 1 00000100:00100000:2.0:1789760763.251796:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747536655984 time 747536395568 state 0x1 ref 1 00000100:00100000:2.0:1789760763.251821:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747536655984 time 747572538109 state 0x0 ref 1 00000100:00100000:2.0:1789760763.251832:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495872128:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760763.251896:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25288704, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc809 00010000:00010000:2.0:1789760763.251905:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454709e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.251921:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760763.251925:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760763.251931:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454709e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.251945:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454709e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760763.277791:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973491 is committed 00002000:00010000:3.0:1789760763.277947:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25288704 -> 25292800 00010000:00010000:3.0:1789760763.277965:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454709e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.277977:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454709e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.277985:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454709e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.277992:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760763.277997:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454709e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.278007:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454709e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.278027:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454709e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.278034:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454709e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.278158:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495872128:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26322us (26878us total) trans 4294973491 rc 0/0 00000100:00100000:3.0:1789760763.278169:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6203 00000100:00100000:3.0:1789760763.278179:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747536655984 time 747572538109 state 0x2 ref 0 00000100:00100000:3.0:1789760763.293413:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760763.293837:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495872256 00000100:00100000:0.0:1789760763.293847:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495872256 00000100:00100000:0.0:1789760763.293975:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747536655984 time 747572538109 state 0x0 ref 1 00000100:00100000:0.0:1789760763.293993:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747572798525 time 747572538109 state 0x1 ref 1 00000100:00100000:0.0:1789760763.294084:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747572798525 time 747614798901 state 0x0 ref 1 00000100:00100000:0.0:1789760763.294102:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495872256:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760763.294150:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25292800, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc809 00010000:00010000:0.0:1789760763.294159:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545470a5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.294174:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:0.0:1789760763.294178:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.294185:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545470a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.294199:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545470a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760763.334094:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973492 is committed 00002000:00010000:0.0:1789760763.334190:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25292800 -> 25296896 00010000:00010000:0.0:1789760763.334209:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545470a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.334223:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545470a5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.334232:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545470a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.334240:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760763.334245:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545470a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.334255:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545470a5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.334277:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545470a5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.334317:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545470a5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.334513:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495872256:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40410us (41106us total) trans 4294973492 rc 0/0 00000100:00100000:0.0:1789760763.334526:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6204 00000100:00100000:0.0:1789760763.334536:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747572798525 time 747614798901 state 0x2 ref 0 00000100:00100000:0.0:1789760763.338790:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760763.339049:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495872384 00000100:00100000:2.0:1789760763.339061:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495872384 00000100:00100000:2.0:1789760763.339115:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747572798525 time 747614798901 state 0x0 ref 1 00000100:00100000:2.0:1789760763.339131:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747615059317 time 747614798901 state 0x1 ref 1 00000100:00100000:2.0:1789760763.339213:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747615059317 time 747659929388 state 0x0 ref 1 00000100:00100000:2.0:1789760763.339226:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495872384:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760763.339320:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25296896, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc809 00010000:00010000:2.0:1789760763.339330:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545470ac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.339345:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760763.339349:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760763.339355:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545470ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.339368:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545470ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.378111:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25296896 -> 25300992 00010000:00010000:3.0:1789760763.378134:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545470ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.378147:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545470ac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.378155:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545470ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.378167:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760763.378172:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545470ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.378180:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545470ac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.378201:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545470ac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.378209:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545470ac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.378332:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495872384:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39102us (39547us total) trans 4294973493 rc 0/0 00000100:00100000:3.0:1789760763.378342:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6205 00000100:00100000:3.0:1789760763.378352:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747615059317 time 747659929388 state 0x2 ref 0 00000001:00080000:1.0:1789760763.378842:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973493 is committed 00000100:00100000:2.0:1789760763.382606:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760763.382757:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495872512 00000100:00100000:3.0:1789760763.382767:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495872512 00000100:00100000:3.0:1789760763.382820:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747615059317 time 747659929388 state 0x0 ref 1 00000100:00100000:3.0:1789760763.382836:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747660189804 time 747659929388 state 0x1 ref 1 00000100:00100000:3.0:1789760763.382918:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747660189804 time 747703633483 state 0x0 ref 1 00000100:00100000:3.0:1789760763.382934:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495872512:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760763.382981:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25300992, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc809 00010000:00010000:3.0:1789760763.382989:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545470b3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.383004:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:3.0:1789760763.383009:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.383016:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545470b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.383029:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545470b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760763.416586:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973494 is committed 00002000:00010000:1.0:1789760763.416677:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25300992 -> 25305088 00010000:00010000:1.0:1789760763.416693:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545470b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.416705:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545470b3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.416714:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545470b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.416721:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760763.416726:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545470b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.416735:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545470b3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.416753:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545470b3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.416761:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545470b3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.416852:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495872512:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33916us (34251us total) trans 4294973494 rc 0/0 00000100:00100000:1.0:1789760763.416868:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6206 00000100:00100000:1.0:1789760763.416886:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747660189804 time 747703633483 state 0x2 ref 0 00000100:00100000:3.0:1789760763.421414:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760763.421449:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495872640 00000100:00100000:0.0:1789760763.421459:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495872640 00000100:00100000:0.0:1789760763.421521:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747660189804 time 747703633483 state 0x0 ref 1 00000100:00100000:0.0:1789760763.421544:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747703893899 time 747703633483 state 0x1 ref 1 00000100:00100000:0.0:1789760763.421569:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747703893899 time 747742286380 state 0x0 ref 1 00000100:00100000:0.0:1789760763.421581:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495872640:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760763.421625:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25305088, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc809 00010000:00010000:0.0:1789760763.421634:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545470ba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.421655:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:0.0:1789760763.421660:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.421666:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545470ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.421679:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545470ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760763.464577:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25305088 -> 25309184 00000001:00080000:1.0:1789760763.464587:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973495 is committed 00010000:00010000:2.0:1789760763.464597:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545470ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.464609:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545470ba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.464619:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545470ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.464626:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760763.464631:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545470ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.464646:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545470ba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.464666:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545470ba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.464672:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x49367483545470ba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760763.464754:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495872640:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43169us (43345us total) trans 4294973495 rc 0/0 00000100:00100000:2.0:1789760763.464765:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6207 00000100:00100000:2.0:1789760763.464775:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747703893899 time 747742286380 state 0x2 ref 0 00000100:00100000:2.0:1789760763.474070:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760763.474191:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495872768 00000100:00100000:3.0:1789760763.474202:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495872768 00000100:00100000:3.0:1789760763.474256:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747703893899 time 747742286380 state 0x0 ref 1 00000100:00100000:3.0:1789760763.474271:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747742546796 time 747742286380 state 0x1 ref 1 00000100:00100000:1.0:1789760763.474568:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747742546796 time 747795281997 state 0x0 ref 1 00000100:00100000:1.0:1789760763.474585:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495872768:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760763.474638:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25309184, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc809 00010000:00010000:1.0:1789760763.474648:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545470c1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.474666:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760763.474670:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.474677:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545470c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.474693:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545470c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760763.508649:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973496 is committed 00002000:00010000:1.0:1789760763.508727:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25309184 -> 25313280 00010000:00010000:1.0:1789760763.508741:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545470c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.508753:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545470c1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.508762:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545470c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.508775:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760763.508781:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545470c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.508790:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545470c1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.508809:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545470c1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.508817:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545470c1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.509016:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495872768:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34430us (34950us total) trans 4294973496 rc 0/0 00000100:00100000:1.0:1789760763.509027:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6208 00000100:00100000:1.0:1789760763.509037:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747742546796 time 747795281997 state 0x2 ref 0 00000100:00100000:3.0:1789760763.513108:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760763.513191:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495872896 00000100:00100000:1.0:1789760763.513209:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495872896 00000100:00100000:1.0:1789760763.513266:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747742546796 time 747795281997 state 0x0 ref 1 00000100:00100000:1.0:1789760763.513321:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747795542413 time 747795281997 state 0x1 ref 1 00000100:00100000:1.0:1789760763.513344:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747795542413 time 747834061952 state 0x0 ref 1 00000100:00100000:1.0:1789760763.513359:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495872896:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760763.513410:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25313280, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc809 00010000:00010000:1.0:1789760763.513418:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545470c8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.513433:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760763.513437:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.513443:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545470c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.513456:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545470c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760763.540796:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25313280 -> 25317376 00010000:00010000:1.0:1789760763.540818:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545470c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.540832:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545470c8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.540841:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545470c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.540848:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760763.540852:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545470c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.540871:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545470c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.540899:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545470c8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.540907:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545470c8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.541000:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495872896:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27638us (27896us total) trans 4294973497 rc 0/0 00000100:00100000:1.0:1789760763.541011:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6209 00000100:00100000:1.0:1789760763.541021:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747795542413 time 747834061952 state 0x2 ref 0 00000001:00080000:0.0:1789760763.541777:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973497 is committed 00000100:00100000:3.0:1789760763.546010:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760763.546128:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495873024 00000100:00100000:2.0:1789760763.546138:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495873024 00000100:00100000:2.0:1789760763.546190:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747795542413 time 747834061952 state 0x0 ref 1 00000100:00100000:2.0:1789760763.546206:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747834322368 time 747834061952 state 0x1 ref 1 00000100:00100000:2.0:1789760763.546372:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747834322368 time 747866993876 state 0x0 ref 1 00000100:00100000:2.0:1789760763.546387:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495873024:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760763.546434:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25317376, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc809 00010000:00010000:2.0:1789760763.546445:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545470cf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.546461:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760763.546465:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760763.546472:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545470cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.546486:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545470cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760763.581219:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973498 is committed 00002000:00010000:1.0:1789760763.581353:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25317376 -> 25321472 00010000:00010000:1.0:1789760763.581373:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545470cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.581387:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545470cf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.581396:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545470cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.581403:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760763.581408:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545470cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.581416:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545470cf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.581434:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545470cf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.581441:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545470cf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.581529:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495873024:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35139us (35523us total) trans 4294973498 rc 0/0 00000100:00100000:1.0:1789760763.581540:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6210 00000100:00100000:1.0:1789760763.581548:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747834322368 time 747866993876 state 0x2 ref 0 00000100:00100000:2.0:1789760763.590980:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760763.591064:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495873152 00000100:00100000:1.0:1789760763.591075:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495873152 00000100:00100000:1.0:1789760763.591134:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747834322368 time 747866993876 state 0x0 ref 1 00000100:00100000:1.0:1789760763.591149:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747867254292 time 747866993876 state 0x1 ref 1 00000100:00100000:1.0:1789760763.591184:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747867254292 time 747911900957 state 0x0 ref 1 00000100:00100000:1.0:1789760763.591197:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495873152:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760763.591249:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25321472, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc809 00010000:00010000:1.0:1789760763.591260:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545470d6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.591277:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760763.591326:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.591334:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545470d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.591349:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545470d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760763.630328:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973499 is committed 00002000:00010000:0.0:1789760763.630502:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25321472 -> 25325568 00010000:00010000:0.0:1789760763.630519:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545470d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.630532:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545470d6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.630542:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545470d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.630550:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760763.630556:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545470d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.630567:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545470d6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.630593:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545470d6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.630601:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545470d6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.630694:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495873152:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39494us (39718us total) trans 4294973499 rc 0/0 00000100:00100000:0.0:1789760763.630705:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6211 00000100:00100000:0.0:1789760763.630715:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747867254292 time 747911900957 state 0x2 ref 0 00000100:00100000:3.0:1789760763.642666:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760763.644133:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495873280 00000100:00100000:1.0:1789760763.644146:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495873280 00000100:00100000:1.0:1789760763.644209:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747867254292 time 747911900957 state 0x0 ref 1 00000100:00100000:1.0:1789760763.644224:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747912161373 time 747911900957 state 0x1 ref 1 00000100:00100000:1.0:1789760763.644259:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747912161373 time 747964975843 state 0x0 ref 1 00000100:00100000:1.0:1789760763.644272:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495873280:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760763.644335:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25325568, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc809 00010000:00010000:1.0:1789760763.644345:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545470dd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.644361:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760763.644365:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.644373:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545470dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.644386:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545470dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.678495:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789760763.678517:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff929005d87800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520311936:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760763.678588:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760763.678654:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520311936 00000100:00100000:0.0:1789760763.678662:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520311936 00000100:00100000:0.0:1789760763.678702:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 208 00000100:00100000:0.0:1789760763.678711:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520311936:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789760763.678743:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760763.678785:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520311936:12345-0@lo:13:osp-pre-1-0.0 Request processed in 74us (197us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760763.678792:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 208 00000100:00100000:0.0:1789760763.678834:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff929005d87800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520311936:0@lo:13:osp-pre-1-0.0 00000001:00080000:0.0:1789760763.696089:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973500 is committed 00002000:00010000:2.0:1789760763.696208:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25325568 -> 25329664 00002000:00010000:2.0:1789760763.696213:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760762 -> 1789760763 00002000:00010000:2.0:1789760763.696215:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760762 -> 1789760763 00010000:00010000:2.0:1789760763.696228:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545470dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.696242:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545470dd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.696251:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545470dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.696258:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760763.696263:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545470dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.696399:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545470dd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.696424:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545470dd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.696432:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545470dd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760763.696557:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495873280:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52282us (53896us total) trans 4294973500 rc 0/0 00000100:00100000:2.0:1789760763.696567:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6212 00000100:00100000:2.0:1789760763.696576:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747912161373 time 747964975843 state 0x2 ref 0 00000100:00100000:0.0:1789760763.700525:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760763.700607:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495873408 00000100:00100000:1.0:1789760763.700617:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495873408 00000100:00100000:1.0:1789760763.700670:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747912161373 time 747964975843 state 0x0 ref 1 00000100:00100000:1.0:1789760763.700686:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747965236259 time 747964975843 state 0x1 ref 1 00000100:00100000:1.0:1789760763.700712:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747965236259 time 748021428973 state 0x0 ref 1 00000100:00100000:1.0:1789760763.700724:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495873408:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760763.700775:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25329664, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc809 00010000:00010000:1.0:1789760763.700784:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545470e4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.700799:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760763.700804:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.700810:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545470e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.700824:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545470e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.743328:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760763.744275:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973501 is committed 00002000:00010000:0.0:1789760763.744373:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25329664 -> 25333760 00010000:00010000:0.0:1789760763.744387:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545470e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.744399:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545470e4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.744408:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545470e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.744417:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760763.744422:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545470e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.744432:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545470e4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.744450:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545470e4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.744457:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545470e4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.744591:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495873408:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43866us (44072us total) trans 4294973501 rc 0/0 00000100:00100000:0.0:1789760763.744602:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6213 00000100:00100000:0.0:1789760763.744612:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747965236259 time 748021428973 state 0x2 ref 0 00000100:00100000:0.0:1789760763.752267:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760763.752401:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495873536 00000100:00100000:1.0:1789760763.752411:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495873536 00000100:00100000:1.0:1789760763.752476:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 747965236259 time 748021428973 state 0x0 ref 1 00000100:00100000:1.0:1789760763.752492:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748021689389 time 748021428973 state 0x1 ref 1 00000100:00100000:1.0:1789760763.752522:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748021689389 time 748073238692 state 0x0 ref 1 00000100:00100000:1.0:1789760763.752534:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495873536:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760763.752582:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25333760, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc809 00010000:00010000:1.0:1789760763.752591:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545470eb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.752607:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760763.752611:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.752618:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545470eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.752633:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545470eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760763.789435:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973502 is committed 00002000:00010000:1.0:1789760763.789523:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25333760 -> 25337856 00010000:00010000:1.0:1789760763.789539:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545470eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.789556:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545470eb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.789565:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545470eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.789572:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760763.789578:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545470eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.789586:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545470eb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.789607:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545470eb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.789614:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545470eb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.789963:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495873536:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37203us (37478us total) trans 4294973502 rc 0/0 00000100:00100000:1.0:1789760763.789976:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6214 00000100:00100000:1.0:1789760763.789986:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748021689389 time 748073238692 state 0x2 ref 0 00000100:00100000:3.0:1789760763.800406:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760763.800510:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495873664 00000100:00100000:2.0:1789760763.800521:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495873664 00000100:00100000:2.0:1789760763.800576:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748021689389 time 748073238692 state 0x0 ref 1 00000100:00100000:2.0:1789760763.800592:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748073499108 time 748073238692 state 0x1 ref 1 00000100:00100000:2.0:1789760763.800682:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748073499108 time 748121398578 state 0x0 ref 1 00000100:00100000:2.0:1789760763.800695:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495873664:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760763.800743:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25337856, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc809 00010000:00010000:2.0:1789760763.800752:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545470f2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.800767:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760763.800771:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760763.800777:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545470f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.800789:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545470f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760763.832325:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973503 is committed 00002000:00010000:3.0:1789760763.832467:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25337856 -> 25341952 00010000:00010000:3.0:1789760763.832483:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545470f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.832496:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545470f2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.832505:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545470f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.832513:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760763.832517:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545470f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.832526:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545470f2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.832546:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545470f2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.832554:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545470f2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.832651:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495873664:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31953us (32247us total) trans 4294973503 rc 0/0 00000100:00100000:3.0:1789760763.832664:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6215 00000100:00100000:3.0:1789760763.832674:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748073499108 time 748121398578 state 0x2 ref 0 00000100:00100000:2.0:1789760763.837612:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760763.837723:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495873792 00000100:00100000:0.0:1789760763.837734:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495873792 00000100:00100000:0.0:1789760763.837789:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748073499108 time 748121398578 state 0x0 ref 1 00000100:00100000:0.0:1789760763.837803:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748121658994 time 748121398578 state 0x1 ref 1 00000100:00100000:0.0:1789760763.837959:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748121658994 time 748158673788 state 0x0 ref 1 00000100:00100000:0.0:1789760763.837973:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495873792:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760763.838023:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25341952, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc809 00010000:00010000:0.0:1789760763.838033:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545470f9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.838061:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:0.0:1789760763.838066:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.838073:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545470f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.838089:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545470f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760763.869547:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25341952 -> 25346048 00010000:00010000:1.0:1789760763.869567:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545470f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.869580:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545470f9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.869590:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545470f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.869597:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760763.869602:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545470f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.869611:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545470f9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.869631:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545470f9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.869640:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545470f9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.869734:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495873792:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31758us (32126us total) trans 4294973504 rc 0/0 00000100:00100000:1.0:1789760763.869745:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6216 00000100:00100000:1.0:1789760763.869755:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748121658994 time 748158673788 state 0x2 ref 0 00000001:00080000:3.0:1789760763.870009:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973504 is committed 00000100:00100000:0.0:1789760763.875478:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760763.875747:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495873920 00000100:00100000:1.0:1789760763.875760:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495873920 00000100:00100000:1.0:1789760763.875822:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748121658994 time 748158673788 state 0x0 ref 1 00000100:00100000:1.0:1789760763.875838:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748158934204 time 748158673788 state 0x1 ref 1 00000100:00100000:1.0:1789760763.875882:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748158934204 time 748196589103 state 0x0 ref 1 00000100:00100000:1.0:1789760763.875896:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495873920:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760763.875949:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25346048, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc809 00010000:00010000:1.0:1789760763.875959:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354547100 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.875976:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760763.875980:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.875986:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354547100 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.875999:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354547100 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760763.908360:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973505 is committed 00002000:00010000:0.0:1789760763.908497:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25346048 -> 25350144 00010000:00010000:0.0:1789760763.908582:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354547100 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.908596:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354547100 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.908606:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354547100 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.908614:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760763.908619:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354547100 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.908629:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354547100 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.908649:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354547100 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.908658:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354547100 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.908792:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495873920:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32894us (33320us total) trans 4294973505 rc 0/0 00000100:00100000:0.0:1789760763.908803:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6217 00000100:00100000:0.0:1789760763.908815:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748158934204 time 748196589103 state 0x2 ref 0 00000100:00100000:0.0:1789760763.913669:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760763.913745:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495874048 00000100:00100000:3.0:1789760763.913755:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495874048 00000100:00100000:3.0:1789760763.913821:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748158934204 time 748196589103 state 0x0 ref 1 00000100:00100000:3.0:1789760763.913836:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748196849519 time 748196589103 state 0x1 ref 1 00000100:00100000:3.0:1789760763.913902:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748196849519 time 748234618515 state 0x0 ref 1 00000100:00100000:3.0:1789760763.913916:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495874048:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760763.913963:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25350144, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc809 00010000:00010000:3.0:1789760763.913973:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547107 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.913990:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:3.0:1789760763.913995:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.914001:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547107 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.914016:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547107 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760763.958145:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973506 is committed 00002000:00010000:0.0:1789760763.958501:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25350144 -> 25354240 00010000:00010000:0.0:1789760763.958517:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547107 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.958530:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547107 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.958538:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547107 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.958545:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760763.958550:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547107 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.958559:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547107 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.958578:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547107 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.958586:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547107 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.958680:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495874048:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44761us (45015us total) trans 4294973506 rc 0/0 00000100:00100000:0.0:1789760763.958693:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6218 00000100:00100000:0.0:1789760763.958703:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748196849519 time 748234618515 state 0x2 ref 0 00000100:00100000:2.0:1789760763.962424:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760763.962644:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495874176 00000100:00100000:1.0:1789760763.962654:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495874176 00000100:00100000:1.0:1789760763.962712:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748196849519 time 748234618515 state 0x0 ref 1 00000100:00100000:1.0:1789760763.962730:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748234878931 time 748234618515 state 0x1 ref 1 00000100:00100000:1.0:1789760763.962762:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748234878931 time 748283479005 state 0x0 ref 1 00000100:00100000:1.0:1789760763.962773:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495874176:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760763.962823:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25354240, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc809 00010000:00010000:1.0:1789760763.962832:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454710e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.962849:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760763.962927:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.962934:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454710e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.962949:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454710e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760763.996386:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973507 is committed 00002000:00010000:1.0:1789760763.996477:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25354240 -> 25358336 00010000:00010000:1.0:1789760763.996492:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454710e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.996505:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454710e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.996514:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454710e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.996520:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760763.996525:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454710e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.996534:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454710e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.996553:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454710e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.996560:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x493674835454710e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.996687:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495874176:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33909us (34409us total) trans 4294973507 rc 0/0 00000100:00100000:1.0:1789760763.996698:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6219 00000100:00100000:1.0:1789760763.996708:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748234878931 time 748283479005 state 0x2 ref 0 00000100:00100000:1.0:1789760764.002055:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760764.002172:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495874304 00000100:00100000:0.0:1789760764.002181:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495874304 00000100:00100000:0.0:1789760764.002453:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748234878931 time 748283479005 state 0x0 ref 1 00000100:00100000:0.0:1789760764.002469:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748283739421 time 748283479005 state 0x1 ref 1 00000100:00100000:0.0:1789760764.002532:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748283739421 time 748323248584 state 0x0 ref 1 00000100:00100000:0.0:1789760764.002546:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495874304:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760764.002593:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25358336, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc809 00010000:00010000:0.0:1789760764.002603:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547115 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.002619:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:0.0:1789760764.002624:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.002630:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547115 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.002644:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547115 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760764.038918:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973508 is committed 00002000:00010000:0.0:1789760764.039143:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25358336 -> 25362432 00010000:00010000:0.0:1789760764.039159:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547115 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.039174:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547115 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.039184:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547115 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.039191:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760764.039208:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547115 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.039218:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547115 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.039239:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547115 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.039247:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547115 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.039385:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495874304:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36837us (37411us total) trans 4294973508 rc 0/0 00000100:00100000:0.0:1789760764.039395:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6220 00000100:00100000:0.0:1789760764.039405:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748283739421 time 748323248584 state 0x2 ref 0 00000100:00100000:0.0:1789760764.042560:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760764.042704:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495874432 00000100:00100000:3.0:1789760764.042714:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495874432 00000100:00100000:3.0:1789760764.042768:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748283739421 time 748323248584 state 0x0 ref 1 00000100:00100000:3.0:1789760764.042784:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748323509000 time 748323248584 state 0x1 ref 1 00000100:00100000:3.0:1789760764.042817:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748323509000 time 748363533906 state 0x0 ref 1 00000100:00100000:3.0:1789760764.042830:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495874432:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760764.042885:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25362432, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc809 00010000:00010000:3.0:1789760764.042895:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454711c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.042911:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:3.0:1789760764.042916:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.042923:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454711c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.042937:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454711c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760764.086682:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25362432 -> 25366528 00010000:00010000:0.0:1789760764.086702:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454711c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.086715:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454711c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.086723:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454711c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.086730:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760764.086734:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454711c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.086743:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454711c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.086762:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454711c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.086779:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454711c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760764.086790:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973509 is committed 00000100:00100000:0.0:1789760764.086892:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495874432:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44059us (44336us total) trans 4294973509 rc 0/0 00000100:00100000:0.0:1789760764.087063:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6221 00000100:00100000:0.0:1789760764.087073:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748323509000 time 748363533906 state 0x2 ref 0 00000100:00100000:3.0:1789760764.091018:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760764.092070:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495874560 00000100:00100000:1.0:1789760764.092082:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495874560 00000100:00100000:1.0:1789760764.092137:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748323509000 time 748363533906 state 0x0 ref 1 00000100:00100000:1.0:1789760764.092152:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748363794322 time 748363533906 state 0x1 ref 1 00000100:00100000:1.0:1789760764.092189:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748363794322 time 748412905087 state 0x0 ref 1 00000100:00100000:1.0:1789760764.092204:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495874560:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760764.092251:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25366528, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc809 00010000:00010000:1.0:1789760764.092260:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547123 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.092306:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760764.092311:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760764.092318:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547123 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.092333:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547123 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760764.123607:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25366528 -> 25370624 00010000:00010000:1.0:1789760764.123628:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547123 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.123641:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547123 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.123650:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547123 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.123657:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760764.123662:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547123 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.123672:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547123 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.123691:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547123 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.123699:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547123 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.123785:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495874560:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31577us (32886us total) trans 4294973510 rc 0/0 00000100:00100000:1.0:1789760764.123796:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6222 00000100:00100000:1.0:1789760764.123805:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748363794322 time 748412905087 state 0x2 ref 0 00000001:00080000:1.0:1789760764.123842:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973510 is committed 00000100:00100000:1.0:1789760764.135840:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760764.136135:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495874688 00000100:00100000:0.0:1789760764.136145:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495874688 00000100:00100000:0.0:1789760764.136201:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748363794322 time 748412905087 state 0x0 ref 1 00000100:00100000:0.0:1789760764.136216:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748413165503 time 748412905087 state 0x1 ref 1 00000100:00100000:0.0:1789760764.136256:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748413165503 time 748456965994 state 0x0 ref 1 00000100:00100000:0.0:1789760764.136270:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495874688:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760764.136340:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25370624, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc809 00010000:00010000:0.0:1789760764.136351:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454712a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.136367:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:0.0:1789760764.136372:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.136378:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454712a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.136392:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454712a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760764.165003:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973511 is committed 00002000:00010000:1.0:1789760764.165110:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25370624 -> 25374720 00010000:00010000:1.0:1789760764.165126:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454712a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.165139:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454712a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.165159:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454712a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.165173:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760764.165178:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454712a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.165186:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454712a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.165207:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454712a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.165215:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454712a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.165347:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495874688:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29075us (29512us total) trans 4294973511 rc 0/0 00000100:00100000:1.0:1789760764.165358:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6223 00000100:00100000:1.0:1789760764.165368:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748413165503 time 748456965994 state 0x2 ref 0 00000100:00100000:1.0:1789760764.170386:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760764.170567:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495874816 00000100:00100000:0.0:1789760764.170579:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495874816 00000100:00100000:0.0:1789760764.170635:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748413165503 time 748456965994 state 0x0 ref 1 00000100:00100000:0.0:1789760764.170651:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748457226410 time 748456965994 state 0x1 ref 1 00000100:00100000:0.0:1789760764.170690:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748457226410 time 748491406517 state 0x0 ref 1 00000100:00100000:0.0:1789760764.170703:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495874816:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760764.170750:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25374720, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc809 00010000:00010000:0.0:1789760764.170758:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547131 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.170775:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:0.0:1789760764.170780:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.170786:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547131 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.170801:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547131 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760764.215712:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973512 is committed 00002000:00010000:3.0:1789760764.215797:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25374720 -> 25378816 00010000:00010000:3.0:1789760764.215817:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547131 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.215831:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547131 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.215846:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547131 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.215964:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760764.215970:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547131 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.215980:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547131 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.216001:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547131 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.216009:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547131 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.216141:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495874816:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45434us (45757us total) trans 4294973512 rc 0/0 00000100:00100000:3.0:1789760764.216152:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6224 00000100:00100000:3.0:1789760764.216164:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748457226410 time 748491406517 state 0x2 ref 0 00000100:00100000:0.0:1789760764.224438:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760764.224535:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495874944 00000100:00100000:1.0:1789760764.224545:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495874944 00000100:00100000:1.0:1789760764.224598:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748457226410 time 748491406517 state 0x0 ref 1 00000100:00100000:1.0:1789760764.224614:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748491666933 time 748491406517 state 0x1 ref 1 00000100:00100000:1.0:1789760764.224648:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748491666933 time 748545364859 state 0x0 ref 1 00000100:00100000:1.0:1789760764.224661:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01c8d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495874944:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760764.224711:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25378816, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc809 00010000:00010000:1.0:1789760764.224721:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547138 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.224737:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760764.224741:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760764.224748:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547138 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.224764:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547138 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760764.261350:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973513 is committed 00002000:00010000:1.0:1789760764.261450:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25378816 -> 25382912 00010000:00010000:1.0:1789760764.261465:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547138 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.261478:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547138 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.261487:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547138 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.261494:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760764.261499:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547138 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.261508:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547138 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.261527:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547138 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.261536:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547138 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.261630:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01c8d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495874944:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36967us (37197us total) trans 4294973513 rc 0/0 00000100:00100000:1.0:1789760764.261641:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6225 00000100:00100000:1.0:1789760764.261651:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748491666933 time 748545364859 state 0x2 ref 0 00000100:00100000:1.0:1789760764.265191:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760764.265327:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495875072 00000100:00100000:0.0:1789760764.265337:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495875072 00000100:00100000:0.0:1789760764.265391:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748491666933 time 748545364859 state 0x0 ref 1 00000100:00100000:0.0:1789760764.265406:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748545625275 time 748545364859 state 0x1 ref 1 00000100:00100000:0.0:1789760764.265428:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748545625275 time 748586144994 state 0x0 ref 1 00000100:00100000:0.0:1789760764.265441:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495875072:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760764.265495:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25382912, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc809 00010000:00010000:0.0:1789760764.265504:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454713f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.265519:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:0.0:1789760764.265524:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.265531:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454713f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.265545:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454713f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760764.320006:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973514 is committed 00002000:00010000:0.0:1789760764.320059:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25382912 -> 25387008 00010000:00010000:0.0:1789760764.320074:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454713f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.320086:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454713f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.320095:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454713f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.320103:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760764.320108:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454713f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.320122:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454713f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.320142:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454713f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.320150:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454713f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.320242:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495875072:12345-192.168.203.9@tcp:4:dd.0 Request processed in 54799us (55055us total) trans 4294973514 rc 0/0 00000100:00100000:0.0:1789760764.320252:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6226 00000100:00100000:0.0:1789760764.320263:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748545625275 time 748586144994 state 0x2 ref 0 00000100:00100000:1.0:1789760764.332880:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760764.332989:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495875200 00000100:00100000:3.0:1789760764.332999:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495875200 00000100:00100000:3.0:1789760764.333052:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748545625275 time 748586144994 state 0x0 ref 1 00000100:00100000:3.0:1789760764.333065:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748586405410 time 748586144994 state 0x1 ref 1 00000100:00100000:3.0:1789760764.333100:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748586405410 time 748653817539 state 0x0 ref 1 00000100:00100000:3.0:1789760764.333113:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04575500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495875200:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760764.333158:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25387008, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc809 00010000:00010000:3.0:1789760764.333168:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547146 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.333186:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:3.0:1789760764.333190:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.333196:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547146 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.333210:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547146 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760764.370859:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973515 is committed 00002000:00010000:1.0:1789760764.371226:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25387008 -> 25391104 00010000:00010000:1.0:1789760764.371242:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547146 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.371528:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547146 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.371542:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547146 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.371551:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760764.371556:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547146 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.371565:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547146 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.371584:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547146 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.371592:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547146 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.371670:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04575500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495875200:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38555us (38804us total) trans 4294973515 rc 0/0 00000100:00100000:1.0:1789760764.371680:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6227 00000100:00100000:1.0:1789760764.371689:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748586405410 time 748653817539 state 0x2 ref 0 00000100:00100000:1.0:1789760764.375420:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760764.375600:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495875328 00000100:00100000:2.0:1789760764.375614:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495875328 00000100:00100000:2.0:1789760764.375668:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748586405410 time 748653817539 state 0x0 ref 1 00000100:00100000:2.0:1789760764.375682:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748654077955 time 748653817539 state 0x1 ref 1 00000100:00100000:2.0:1789760764.375794:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748654077955 time 748696509502 state 0x0 ref 1 00000100:00100000:2.0:1789760764.375807:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04577b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495875328:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760764.375856:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25391104, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc809 00010000:00010000:2.0:1789760764.375866:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454714d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.375901:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760764.375905:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.375912:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454714d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.375925:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454714d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.406223:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760764.406266:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760764.406327:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760764.406361:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495875712 00000100:00100000:1.0:1789760764.406370:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495875712 00000100:00100000:1.0:1789760764.406428:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495875584 00000100:00100000:1.0:1789760764.406431:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495875584 00000100:00100000:1.0:1789760764.406457:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 505 00000100:00100000:1.0:1789760764.406467:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2356:x1876699495875712:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:1.0:1789760764.406537:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2356:x1876699495875712:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 71us (269us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760764.406544:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 505 00000100:00100000:1.0:1789760764.406560:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 506 00000100:00100000:0.0:1789760764.406566:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495875456 00000100:00100000:1.0:1789760764.406567:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:17022965-1e4f-46fb-915c-61730096b8e2+8:2356:x1876699495875584:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:0.0:1789760764.406574:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495875456 00000100:00100000:0.0:1789760764.406618:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 410 00000100:00100000:0.0:1789760764.406626:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929012312050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2356:x1876699495875456:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:1.0:1789760764.406742:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04574700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:17022965-1e4f-46fb-915c-61730096b8e2+8:2356:x1876699495875584:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 175us (414us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760764.406749:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 506 00000100:00080000:1.0:1789760764.406765:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760764.407451:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929012312050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2356:x1876699495875456:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 822us (1232us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760764.407460:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 410 00000001:00080000:0.0:1789760764.438800:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973516 is committed 00002000:00010000:2.0:1789760764.438951:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25391104 -> 25395200 00010000:00010000:2.0:1789760764.438968:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454714d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.438980:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454714d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.438988:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454714d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.438996:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760764.439000:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454714d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.439016:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454714d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.439373:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454714d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.439382:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454714d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.439611:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04577b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495875328:12345-192.168.203.9@tcp:4:dd.0 Request processed in 63799us (64193us total) trans 4294973516 rc 0/0 00000100:00100000:3.0:1789760764.439624:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6228 00000100:00100000:3.0:1789760764.439634:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748654077955 time 748696509502 state 0x2 ref 0 00000100:00100000:1.0:1789760764.443179:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760764.443533:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495875840 00000100:00100000:0.0:1789760764.443543:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495875840 00000100:00100000:0.0:1789760764.443597:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748654077955 time 748696509502 state 0x0 ref 1 00000100:00100000:0.0:1789760764.443612:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748696769918 time 748696509502 state 0x1 ref 1 00000100:00100000:0.0:1789760764.443681:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748696769918 time 748764397020 state 0x0 ref 1 00000100:00100000:0.0:1789760764.443694:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495875840:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760764.443744:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25395200, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc809 00010000:00010000:0.0:1789760764.443753:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547154 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.443770:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:0.0:1789760764.443776:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.443783:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547154 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.443797:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547154 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760764.476136:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973517 is committed 00002000:00010000:1.0:1789760764.476313:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25395200 -> 25399296 00010000:00010000:1.0:1789760764.476329:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547154 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.476341:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547154 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.476350:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547154 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.476357:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760764.476361:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547154 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.476375:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547154 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.476394:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547154 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.476401:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547154 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.476485:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495875840:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32789us (33314us total) trans 4294973517 rc 0/0 00000100:00100000:1.0:1789760764.476495:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6229 00000100:00100000:1.0:1789760764.476504:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748696769918 time 748764397020 state 0x2 ref 0 00000100:00100000:1.0:1789760764.479663:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760764.480049:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495875968 00000100:00100000:2.0:1789760764.480059:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495875968 00000100:00100000:2.0:1789760764.480113:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748696769918 time 748764397020 state 0x0 ref 1 00000100:00100000:2.0:1789760764.480127:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748764657436 time 748764397020 state 0x1 ref 1 00000100:00100000:2.0:1789760764.480154:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748764657436 time 748800871865 state 0x0 ref 1 00000100:00100000:2.0:1789760764.480165:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495875968:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760764.480218:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25399296, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc809 00010000:00010000:2.0:1789760764.480226:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454715b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.480242:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760764.480246:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.480251:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454715b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.480265:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454715b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760764.515618:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973518 is committed 00002000:00010000:1.0:1789760764.515743:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25399296 -> 25403392 00010000:00010000:1.0:1789760764.515758:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454715b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.515779:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454715b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.515787:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454715b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.515799:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760764.515803:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454715b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.515812:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454715b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.515828:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454715b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.515835:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454715b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.516087:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495875968:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35917us (36426us total) trans 4294973518 rc 0/0 00000100:00100000:1.0:1789760764.516098:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6230 00000100:00100000:1.0:1789760764.516108:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748764657436 time 748800871865 state 0x2 ref 0 00000100:00100000:0.0:1789760764.520209:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760764.520580:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495876096 00000100:00100000:3.0:1789760764.520591:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495876096 00000100:00100000:2.0:1789760764.520751:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748764657436 time 748800871865 state 0x0 ref 1 00000100:00100000:2.0:1789760764.520769:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748801132281 time 748800871865 state 0x1 ref 1 00000100:00100000:2.0:1789760764.520831:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748801132281 time 748841546772 state 0x0 ref 1 00000100:00100000:2.0:1789760764.520857:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495876096:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760764.520913:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25403392, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc809 00010000:00010000:2.0:1789760764.520922:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547162 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.520937:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760764.520942:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.520949:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547162 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.520963:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547162 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760764.550098:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973519 is committed 00002000:00010000:0.0:1789760764.550217:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25403392 -> 25407488 00010000:00010000:0.0:1789760764.550232:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547162 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.550246:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547162 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.550255:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547162 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.550268:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760764.550273:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547162 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.550333:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547162 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.550351:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547162 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.550358:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547162 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.550464:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495876096:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29610us (30269us total) trans 4294973519 rc 0/0 00000100:00100000:0.0:1789760764.550474:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6231 00000100:00100000:0.0:1789760764.550483:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748801132281 time 748841546772 state 0x2 ref 0 00000100:00100000:0.0:1789760764.558627:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760764.558727:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495876224 00000100:00100000:1.0:1789760764.558737:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495876224 00000100:00100000:1.0:1789760764.558788:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748801132281 time 748841546772 state 0x0 ref 1 00000100:00100000:1.0:1789760764.558802:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748841807188 time 748841546772 state 0x1 ref 1 00000100:00100000:1.0:1789760764.558888:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748841807188 time 748879603989 state 0x0 ref 1 00000100:00100000:1.0:1789760764.558909:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495876224:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760764.558961:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25407488, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc809 00010000:00010000:1.0:1789760764.558970:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547169 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.558987:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760764.558991:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760764.558997:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547169 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.559009:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547169 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760764.589553:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973520 is committed 00002000:00010000:0.0:1789760764.589761:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25407488 -> 25411584 00010000:00010000:0.0:1789760764.589778:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547169 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.589791:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547169 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.589806:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547169 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.589814:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760764.589820:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547169 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.589830:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547169 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.589918:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547169 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.589927:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547169 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.590060:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495876224:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31156us (31438us total) trans 4294973520 rc 0/0 00000100:00100000:0.0:1789760764.590070:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6232 00000100:00100000:0.0:1789760764.590080:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748841807188 time 748879603989 state 0x2 ref 0 00000100:00100000:1.0:1789760764.593776:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760764.593933:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495876352 00000100:00100000:2.0:1789760764.593942:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495876352 00000100:00100000:2.0:1789760764.593998:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748841807188 time 748879603989 state 0x0 ref 1 00000100:00100000:2.0:1789760764.594017:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748879864405 time 748879603989 state 0x1 ref 1 00000100:00100000:2.0:1789760764.594049:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748879864405 time 748914766198 state 0x0 ref 1 00000100:00100000:2.0:1789760764.594062:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495876352:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760764.594111:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25411584, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc809 00010000:00010000:2.0:1789760764.594120:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547170 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.594138:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760764.594143:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.594149:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547170 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.594163:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547170 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760764.621838:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973521 is committed 00002000:00010000:2.0:1789760764.622024:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25411584 -> 25415680 00010000:00010000:2.0:1789760764.622040:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547170 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.622053:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547170 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.622063:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547170 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.622076:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760764.622081:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547170 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.622091:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547170 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.622110:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547170 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.622118:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547170 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760764.622321:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495876352:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28217us (28510us total) trans 4294973521 rc 0/0 00000100:00100000:2.0:1789760764.622332:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6233 00000100:00100000:2.0:1789760764.622343:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748879864405 time 748914766198 state 0x2 ref 0 00000100:00100000:3.0:1789760764.628502:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760764.628818:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495876480 00000100:00100000:2.0:1789760764.628828:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495876480 00000100:00100000:2.0:1789760764.628900:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748879864405 time 748914766198 state 0x0 ref 1 00000100:00100000:2.0:1789760764.628915:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748915026614 time 748914766198 state 0x1 ref 1 00000100:00100000:2.0:1789760764.628939:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748915026614 time 748949656499 state 0x0 ref 1 00000100:00100000:2.0:1789760764.628955:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495876480:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760764.629005:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25415680, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc809 00010000:00010000:2.0:1789760764.629014:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547177 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.629032:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760764.629036:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.629043:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547177 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.629056:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547177 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760764.675488:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973522 is committed 00002000:00010000:2.0:1789760764.675634:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25415680 -> 25419776 00002000:00010000:2.0:1789760764.675638:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760763 -> 1789760764 00002000:00010000:2.0:1789760764.675641:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760763 -> 1789760764 00010000:00010000:2.0:1789760764.675654:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547177 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.675667:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547177 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.675677:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547177 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.675689:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760764.675694:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547177 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.675703:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547177 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.675722:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547177 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.675730:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547177 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760764.675935:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495876480:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46977us (47436us total) trans 4294973522 rc 0/0 00000100:00100000:2.0:1789760764.676006:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6234 00000100:00100000:2.0:1789760764.676019:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748915026614 time 748949656499 state 0x2 ref 0 00000100:00100000:1.0:1789760764.679839:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760764.680041:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495876608 00000100:00100000:0.0:1789760764.680051:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495876608 00000100:00100000:0.0:1789760764.680108:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748915026614 time 748949656499 state 0x0 ref 1 00000100:00100000:0.0:1789760764.680124:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748949916915 time 748949656499 state 0x1 ref 1 00000100:00100000:0.0:1789760764.680263:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748949916915 time 749000978124 state 0x0 ref 1 00000100:00100000:0.0:1789760764.681506:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495876608:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760764.681649:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25419776, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc809 00010000:00010000:0.0:1789760764.681660:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454717e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.681678:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760764.681683:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.681689:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454717e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.681706:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454717e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760764.720050:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25419776 -> 25423872 00010000:00010000:1.0:1789760764.720068:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454717e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.720082:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454717e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.720092:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454717e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.720105:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760764.720110:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454717e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.720120:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454717e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.720140:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454717e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.720147:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454717e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.720234:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495876608:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39956us (40400us total) trans 4294973523 rc 0/0 00000100:00100000:1.0:1789760764.720245:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6235 00000100:00100000:1.0:1789760764.720255:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748949916915 time 749000978124 state 0x2 ref 0 00000001:00080000:2.0:1789760764.721146:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973523 is committed 00000100:00100000:1.0:1789760764.726569:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760764.726731:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495876736 00000100:00100000:2.0:1789760764.726742:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495876736 00000100:00100000:2.0:1789760764.726799:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 748949916915 time 749000978124 state 0x0 ref 1 00000100:00100000:2.0:1789760764.726814:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749001238540 time 749000978124 state 0x1 ref 1 00000100:00100000:2.0:1789760764.726855:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749001238540 time 749047570924 state 0x0 ref 1 00000100:00100000:2.0:1789760764.726880:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495876736:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760764.727033:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25423872, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc809 00010000:00010000:2.0:1789760764.727043:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547185 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.727059:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760764.727064:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.727070:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547185 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.727084:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547185 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760764.759433:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973524 is committed 00002000:00010000:2.0:1789760764.759508:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25423872 -> 25427968 00010000:00010000:2.0:1789760764.759523:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547185 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.759536:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547185 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.759544:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547185 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.759551:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760764.759556:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547185 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.759565:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547185 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.759584:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547185 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.759591:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547185 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760764.759685:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495876736:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32815us (33120us total) trans 4294973524 rc 0/0 00000100:00100000:2.0:1789760764.759695:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6236 00000100:00100000:2.0:1789760764.759705:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749001238540 time 749047570924 state 0x2 ref 0 00010000:00010000:2.0:1789760764.766895:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760764.771691:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760764.771941:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495876864 00000100:00100000:2.0:1789760764.771951:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495876864 00000100:00100000:2.0:1789760764.772004:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749001238540 time 749047570924 state 0x0 ref 1 00000100:00100000:2.0:1789760764.772019:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749047831340 time 749047570924 state 0x1 ref 1 00000100:00100000:2.0:1789760764.772050:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749047831340 time 749092766762 state 0x0 ref 1 00000100:00100000:2.0:1789760764.772064:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495876864:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760764.772132:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25427968, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc809 00010000:00010000:2.0:1789760764.772143:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454718c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.772161:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760764.772166:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.772175:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454718c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.772190:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454718c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760764.802641:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973525 is committed 00002000:00010000:3.0:1789760764.802780:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25427968 -> 25432064 00010000:00010000:3.0:1789760764.802796:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454718c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.802809:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454718c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.802822:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454718c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.802830:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760764.802834:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454718c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.802843:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454718c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.802881:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454718c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.802897:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454718c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.802986:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495876864:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30921us (31301us total) trans 4294973525 rc 0/0 00000100:00100000:3.0:1789760764.802996:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6237 00000100:00100000:3.0:1789760764.803006:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749047831340 time 749092766762 state 0x2 ref 0 00000100:00100000:1.0:1789760764.808309:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760764.808412:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495876992 00000100:00100000:0.0:1789760764.808423:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495876992 00000100:00100000:0.0:1789760764.808485:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749047831340 time 749092766762 state 0x0 ref 1 00000100:00100000:0.0:1789760764.808502:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749093027178 time 749092766762 state 0x1 ref 1 00000100:00100000:0.0:1789760764.808533:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749093027178 time 749129250284 state 0x0 ref 1 00000100:00100000:0.0:1789760764.808547:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903639bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495876992:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760764.808612:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25432064, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc809 00010000:00010000:0.0:1789760764.808622:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547193 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.808638:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760764.808642:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.808648:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547193 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.808662:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547193 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760764.862771:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25432064 -> 25436160 00010000:00010000:0.0:1789760764.862788:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547193 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.862809:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547193 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.862825:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547193 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.862834:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760764.862839:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547193 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.862856:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547193 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.862892:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547193 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.862900:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547193 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.863010:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903639bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495876992:12345-192.168.203.9@tcp:4:dd.0 Request processed in 54460us (54733us total) trans 4294973526 rc 0/0 00000100:00100000:0.0:1789760764.863021:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6238 00000100:00100000:0.0:1789760764.863032:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749093027178 time 749129250284 state 0x2 ref 0 00000001:00080000:0.0:1789760764.863241:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973526 is committed 00000100:00100000:1.0:1789760764.874699:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760764.874932:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495877120 00000100:00100000:2.0:1789760764.874943:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495877120 00000100:00100000:2.0:1789760764.875009:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749093027178 time 749129250284 state 0x0 ref 1 00000100:00100000:2.0:1789760764.875024:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749129510700 time 749129250284 state 0x1 ref 1 00000100:00100000:2.0:1789760764.875054:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749129510700 time 749195771173 state 0x0 ref 1 00000100:00100000:2.0:1789760764.875067:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495877120:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760764.875116:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25436160, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc809 00010000:00010000:2.0:1789760764.875125:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454719a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.875140:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760764.875144:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.875151:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454719a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.875165:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454719a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760764.907154:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25436160 -> 25440256 00010000:00010000:0.0:1789760764.907174:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454719a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760764.907276:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973527 is committed 00010000:00010000:0.0:1789760764.907364:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454719a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.907373:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454719a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.907381:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760764.907386:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454719a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.907400:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454719a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.907421:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454719a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.907429:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454719a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.907520:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495877120:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32452us (32833us total) trans 4294973527 rc 0/0 00000100:00100000:0.0:1789760764.907530:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6239 00000100:00100000:0.0:1789760764.907541:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749129510700 time 749195771173 state 0x2 ref 0 00000100:00100000:1.0:1789760764.912656:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760764.913581:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495877248 00000100:00100000:3.0:1789760764.913592:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495877248 00000100:00100000:3.0:1789760764.913661:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749129510700 time 749195771173 state 0x0 ref 1 00000100:00100000:3.0:1789760764.913676:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749196031589 time 749195771173 state 0x1 ref 1 00000100:00100000:3.0:1789760764.913748:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749196031589 time 749234464591 state 0x0 ref 1 00000100:00100000:3.0:1789760764.913761:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495877248:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760764.913811:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25440256, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc809 00010000:00010000:3.0:1789760764.913831:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545471a1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.914005:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760764.914010:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.914017:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545471a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.914032:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545471a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760764.953771:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25440256 -> 25444352 00010000:00010000:3.0:1789760764.953793:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545471a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.953814:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545471a1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.953838:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545471a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.953852:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760764.953858:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545471a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.953876:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545471a1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.953899:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545471a1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.953907:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545471a1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.954005:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495877248:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40240us (41352us total) trans 4294973528 rc 0/0 00000100:00100000:3.0:1789760764.954017:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6240 00000100:00100000:3.0:1789760764.954029:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749196031589 time 749234464591 state 0x2 ref 0 00000001:00080000:1.0:1789760764.954557:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973528 is committed 00000100:00100000:1.0:1789760764.958828:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760764.959236:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495877376 00000100:00100000:0.0:1789760764.959246:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495877376 00000100:00100000:0.0:1789760764.959326:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749196031589 time 749234464591 state 0x0 ref 1 00000100:00100000:0.0:1789760764.959342:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749234725007 time 749234464591 state 0x1 ref 1 00000100:00100000:0.0:1789760764.959369:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749234725007 time 749280085698 state 0x0 ref 1 00000100:00100000:0.0:1789760764.959382:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495877376:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760764.959430:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25444352, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc809 00010000:00010000:0.0:1789760764.959438:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545471a8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.959455:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760764.959459:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.959466:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545471a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.959479:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545471a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760765.010434:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973529 is committed 00002000:00010000:1.0:1789760765.010625:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25444352 -> 25448448 00010000:00010000:1.0:1789760765.010642:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545471a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.010654:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545471a8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.010663:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545471a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.010670:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760765.010675:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545471a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.010685:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545471a8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.010704:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545471a8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.010712:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545471a8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.010814:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495877376:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51429us (51989us total) trans 4294973529 rc 0/0 00000100:00100000:1.0:1789760765.010825:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6241 00000100:00100000:1.0:1789760765.010834:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749234725007 time 749280085698 state 0x2 ref 0 00000100:00100000:3.0:1789760765.015546:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760765.015659:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495877504 00000100:00100000:0.0:1789760765.015669:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495877504 00000100:00100000:0.0:1789760765.015758:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749234725007 time 749280085698 state 0x0 ref 1 00000100:00100000:0.0:1789760765.015774:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749280346114 time 749280085698 state 0x1 ref 1 00000100:00100000:0.0:1789760765.015803:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749280346114 time 749336519874 state 0x0 ref 1 00000100:00100000:0.0:1789760765.015815:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495877504:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760765.015878:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25448448, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc809 00010000:00010000:0.0:1789760765.015887:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545471af lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.015903:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760765.015908:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760765.015915:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545471af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.015933:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545471af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760765.019562:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760765.051628:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973530 is committed 00002000:00010000:0.0:1789760765.053163:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25448448 -> 25452544 00010000:00010000:0.0:1789760765.053360:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545471af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.053434:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545471af lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.053444:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545471af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.053452:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94aa00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760765.053457:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545471af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.053467:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545471af lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.053486:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545471af lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.053494:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545471af lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.053623:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495877504:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37805us (38081us total) trans 4294973530 rc 0/0 00000100:00100000:0.0:1789760765.053633:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6242 00000100:00100000:0.0:1789760765.053643:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749280346114 time 749336519874 state 0x2 ref 0 00000100:00100000:0.0:1789760765.059704:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760765.059813:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495877632 00000100:00100000:3.0:1789760765.059957:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495877632 00000100:00100000:3.0:1789760765.060088:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749280346114 time 749336519874 state 0x0 ref 1 00000100:00100000:3.0:1789760765.060104:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749336780290 time 749336519874 state 0x1 ref 1 00000100:00100000:3.0:1789760765.060171:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749336780290 time 749380887427 state 0x0 ref 1 00000100:00100000:3.0:1789760765.060194:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495877632:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760765.060242:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25452544, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc809 00010000:00010000:3.0:1789760765.060251:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545471b6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.060267:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760765.060272:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.060317:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545471b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.060335:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545471b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760765.123734:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25452544 -> 25456640 00010000:00010000:1.0:1789760765.123754:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545471b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.123768:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545471b6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.123778:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545471b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.123786:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760765.123791:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545471b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.123800:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545471b6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.123830:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545471b6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.123838:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545471b6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.123944:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495877632:12345-192.168.203.9@tcp:4:dd.0 Request processed in 63756us (64244us total) trans 4294973531 rc 0/0 00000100:00100000:1.0:1789760765.123959:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6243 00000100:00100000:1.0:1789760765.124142:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749336780290 time 749380887427 state 0x2 ref 0 00000001:00080000:0.0:1789760765.125645:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973531 is committed 00000100:00100000:1.0:1789760765.130523:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760765.130751:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495877760 00000100:00100000:0.0:1789760765.130761:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495877760 00000100:00100000:0.0:1789760765.130812:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749336780290 time 749380887427 state 0x0 ref 1 00000100:00100000:0.0:1789760765.130828:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749381147843 time 749380887427 state 0x1 ref 1 00000100:00100000:0.0:1789760765.131014:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749381147843 time 749451729894 state 0x0 ref 1 00000100:00100000:0.0:1789760765.131029:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495877760:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760765.131079:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25456640, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc809 00010000:00010000:0.0:1789760765.131088:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545471bd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.131104:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760765.131118:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760765.131125:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545471bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.131143:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545471bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760765.203490:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973532 is committed 00002000:00010000:1.0:1789760765.206080:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25456640 -> 25460736 00010000:00010000:1.0:1789760765.206099:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545471bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.206112:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545471bd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.206120:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545471bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.206127:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ae00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760765.206132:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545471bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.206142:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545471bd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.206159:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545471bd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.206167:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545471bd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.206271:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495877760:12345-192.168.203.9@tcp:4:dd.0 Request processed in 75241us (75754us total) trans 4294973532 rc 0/0 00000100:00100000:1.0:1789760765.206325:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6244 00000100:00100000:1.0:1789760765.206335:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749381147843 time 749451729894 state 0x2 ref 0 00000100:00100000:0.0:1789760765.211549:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760765.211654:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495877888 00000100:00100000:2.0:1789760765.211663:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495877888 00000100:00100000:2.0:1789760765.211717:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749381147843 time 749451729894 state 0x0 ref 1 00000100:00100000:2.0:1789760765.211732:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749451990310 time 749451729894 state 0x1 ref 1 00000100:00100000:2.0:1789760765.211764:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749451990310 time 749532480930 state 0x0 ref 1 00000100:00100000:2.0:1789760765.211776:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495877888:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760765.211824:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25460736, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc809 00010000:00010000:2.0:1789760765.211834:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545471c4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.211850:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760765.211855:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.211861:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545471c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.211884:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545471c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760765.262096:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973533 is committed 00002000:00010000:2.0:1789760765.262148:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25460736 -> 25464832 00010000:00010000:2.0:1789760765.262166:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545471c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.262178:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545471c4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.262188:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545471c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.262196:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760765.262202:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545471c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.262212:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545471c4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.262232:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545471c4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.262241:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545471c4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760765.262364:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495877888:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50584us (50819us total) trans 4294973533 rc 0/0 00000100:00100000:2.0:1789760765.262374:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6245 00000100:00100000:2.0:1789760765.262383:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749451990310 time 749532480930 state 0x2 ref 0 00000100:00100000:0.0:1789760765.270366:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760765.270490:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495878016 00000100:00100000:1.0:1789760765.270500:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495878016 00000100:00100000:1.0:1789760765.270553:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749451990310 time 749532480930 state 0x0 ref 1 00000100:00100000:1.0:1789760765.270569:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749532741346 time 749532480930 state 0x1 ref 1 00000100:00100000:1.0:1789760765.270712:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749532741346 time 749591427535 state 0x0 ref 1 00000100:00100000:1.0:1789760765.270726:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495878016:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760765.270776:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25464832, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc809 00010000:00010000:1.0:1789760765.270785:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545471cb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.270800:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760765.270805:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760765.270812:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545471cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.270826:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545471cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760765.300672:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973534 is committed 00002000:00010000:1.0:1789760765.300760:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25464832 -> 25468928 00010000:00010000:1.0:1789760765.300775:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545471cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.300788:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545471cb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.300797:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545471cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.300804:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760765.300809:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545471cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.300818:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545471cb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.300837:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545471cb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.300854:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545471cb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.300985:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495878016:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30258us (30803us total) trans 4294973534 rc 0/0 00000100:00100000:1.0:1789760765.301008:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6246 00000100:00100000:1.0:1789760765.301018:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749532741346 time 749591427535 state 0x2 ref 0 00000100:00100000:1.0:1789760765.307786:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760765.307904:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495878144 00000100:00100000:0.0:1789760765.307914:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495878144 00000100:00100000:0.0:1789760765.307965:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749532741346 time 749591427535 state 0x0 ref 1 00000100:00100000:0.0:1789760765.307981:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749591687951 time 749591427535 state 0x1 ref 1 00000100:00100000:0.0:1789760765.308136:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749591687951 time 749628851276 state 0x0 ref 1 00000100:00100000:0.0:1789760765.308149:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495878144:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760765.308199:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25468928, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc809 00010000:00010000:0.0:1789760765.308209:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471d2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.308225:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760765.308230:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760765.308237:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.308249:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760765.345595:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973535 is committed 00002000:00010000:0.0:1789760765.345696:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25468928 -> 25473024 00010000:00010000:0.0:1789760765.345715:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.345728:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471d2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.345737:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.345745:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760765.345750:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.345758:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471d2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.345778:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471d2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.345786:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471d2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.346074:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495878144:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37922us (38292us total) trans 4294973535 rc 0/0 00000100:00100000:0.0:1789760765.346088:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6247 00000100:00100000:0.0:1789760765.346099:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749591687951 time 749628851276 state 0x2 ref 0 00000100:00100000:0.0:1789760765.353272:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760765.353816:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495878272 00000100:00100000:2.0:1789760765.353828:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495878272 00000100:00100000:2.0:1789760765.353890:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749591687951 time 749628851276 state 0x0 ref 1 00000100:00100000:2.0:1789760765.353916:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749629111692 time 749628851276 state 0x1 ref 1 00000100:00100000:2.0:1789760765.354105:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749629111692 time 749674819139 state 0x0 ref 1 00000100:00100000:2.0:1789760765.354119:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495878272:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760765.354179:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25473024, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc809 00010000:00010000:2.0:1789760765.354188:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545471d9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.354204:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760765.354208:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.354215:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545471d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.354229:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545471d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760765.394477:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25473024 -> 25477120 00010000:00010000:0.0:1789760765.394497:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545471d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.394511:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545471d9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.394520:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545471d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.394528:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760765.394533:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545471d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.394543:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545471d9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.394561:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545471d9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.394570:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545471d9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.394663:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495878272:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40541us (41395us total) trans 4294973536 rc 0/0 00000100:00100000:0.0:1789760765.394673:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6248 00000100:00100000:0.0:1789760765.394683:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749629111692 time 749674819139 state 0x2 ref 0 00000001:00080000:2.0:1789760765.394791:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973536 is committed 00000100:00100000:1.0:1789760765.402918:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760765.403020:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495878400 00000100:00100000:3.0:1789760765.403032:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495878400 00000100:00100000:3.0:1789760765.403085:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749629111692 time 749674819139 state 0x0 ref 1 00000100:00100000:3.0:1789760765.403102:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749675079555 time 749674819139 state 0x1 ref 1 00000100:00100000:3.0:1789760765.403138:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749675079555 time 749723854210 state 0x0 ref 1 00000100:00100000:3.0:1789760765.403149:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495878400:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760765.403197:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25477120, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc809 00010000:00010000:3.0:1789760765.403207:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545471e0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.403223:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760765.403228:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.403235:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545471e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.403249:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545471e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760765.438014:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973537 is committed 00002000:00010000:0.0:1789760765.438108:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25477120 -> 25481216 00010000:00010000:0.0:1789760765.438123:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545471e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.438136:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545471e0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.438145:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545471e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.438152:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760765.438157:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545471e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.438166:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545471e0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.438185:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545471e0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.438198:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545471e0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.438321:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495878400:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35169us (35409us total) trans 4294973537 rc 0/0 00000100:00100000:0.0:1789760765.438331:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6249 00000100:00100000:0.0:1789760765.438341:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749675079555 time 749723854210 state 0x2 ref 0 00000100:00100000:3.0:1789760765.442191:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760765.442318:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495878528 00000100:00100000:0.0:1789760765.442327:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495878528 00000100:00100000:0.0:1789760765.442378:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749675079555 time 749723854210 state 0x0 ref 1 00000100:00100000:0.0:1789760765.442394:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749724114626 time 749723854210 state 0x1 ref 1 00000100:00100000:0.0:1789760765.442420:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749724114626 time 749763136884 state 0x0 ref 1 00000100:00100000:0.0:1789760765.442431:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495878528:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760765.442474:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25481216, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc809 00010000:00010000:0.0:1789760765.442482:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471e7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.442499:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760765.442504:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760765.442510:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.442529:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760765.491117:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25481216 -> 25485312 00010000:00010000:0.0:1789760765.491137:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.491149:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471e7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.491158:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.491165:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760765.491170:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.491178:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471e7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.491195:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471e7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.491202:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471e7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.491330:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495878528:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48894us (49143us total) trans 4294973538 rc 0/0 00000100:00100000:0.0:1789760765.491342:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6250 00000100:00100000:0.0:1789760765.491352:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749724114626 time 749763136884 state 0x2 ref 0 00000001:00080000:3.0:1789760765.491869:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973538 is committed 00000100:00100000:3.0:1789760765.497999:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760765.498172:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495878656 00000100:00100000:0.0:1789760765.498182:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495878656 00000100:00100000:0.0:1789760765.498234:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749724114626 time 749763136884 state 0x0 ref 1 00000100:00100000:0.0:1789760765.498248:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749763397300 time 749763136884 state 0x1 ref 1 00000100:00100000:0.0:1789760765.498315:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749763397300 time 749818997224 state 0x0 ref 1 00000100:00100000:0.0:1789760765.498329:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495878656:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760765.498374:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25485312, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc809 00010000:00010000:0.0:1789760765.498384:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471ee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.498400:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760765.498404:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760765.498411:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.498425:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760765.532622:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973539 is committed 00002000:00010000:2.0:1789760765.532738:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25485312 -> 25489408 00010000:00010000:2.0:1789760765.532756:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.532771:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471ee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.532780:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.532788:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760765.532793:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.532801:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471ee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.532821:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471ee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.532829:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545471ee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760765.533055:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495878656:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34724us (35060us total) trans 4294973539 rc 0/0 00000100:00100000:2.0:1789760765.533066:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6251 00000100:00100000:2.0:1789760765.533076:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749763397300 time 749818997224 state 0x2 ref 0 00000100:00100000:2.0:1789760765.539462:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760765.539550:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495878784 00000100:00100000:0.0:1789760765.539559:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495878784 00000100:00100000:0.0:1789760765.539615:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749763397300 time 749818997224 state 0x0 ref 1 00000100:00100000:0.0:1789760765.539630:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749819257640 time 749818997224 state 0x1 ref 1 00000100:00100000:0.0:1789760765.539662:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749819257640 time 749860378739 state 0x0 ref 1 00000100:00100000:0.0:1789760765.539676:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495878784:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760765.539725:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25489408, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc809 00010000:00010000:0.0:1789760765.539734:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545471f5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.539750:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760765.539755:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760765.539760:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545471f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.539773:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545471f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760765.571316:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973540 is committed 00002000:00010000:3.0:1789760765.571420:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25489408 -> 25493504 00010000:00010000:3.0:1789760765.571436:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545471f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.571450:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545471f5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.571466:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545471f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.571473:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760765.571478:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545471f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.571487:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545471f5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.571508:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545471f5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.571515:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545471f5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.571615:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495878784:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31936us (32158us total) trans 4294973540 rc 0/0 00000100:00100000:3.0:1789760765.571625:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6252 00000100:00100000:3.0:1789760765.571636:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749819257640 time 749860378739 state 0x2 ref 0 00000100:00100000:3.0:1789760765.575541:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760765.575657:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495878912 00000100:00100000:0.0:1789760765.575796:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495878912 00000100:00100000:0.0:1789760765.575854:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749819257640 time 749860378739 state 0x0 ref 1 00000100:00100000:0.0:1789760765.575884:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749860639155 time 749860378739 state 0x1 ref 1 00000100:00100000:0.0:1789760765.575943:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749860639155 time 749896658890 state 0x0 ref 1 00000100:00100000:0.0:1789760765.575958:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03096680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495878912:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760765.576017:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25493504, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc809 00010000:00010000:0.0:1789760765.576026:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545471fc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.576044:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760765.576049:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760765.576055:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545471fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.576068:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545471fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760765.609056:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25493504 -> 25497600 00010000:00010000:0.0:1789760765.609077:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545471fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.609091:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545471fc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.609100:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545471fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.609109:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760765.609115:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545471fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.609124:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545471fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.609144:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545471fc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.609152:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545471fc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.609243:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03096680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495878912:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33285us (33708us total) trans 4294973541 rc 0/0 00000100:00100000:0.0:1789760765.609254:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6253 00000100:00100000:0.0:1789760765.609265:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749860639155 time 749896658890 state 0x2 ref 0 00000001:00080000:2.0:1789760765.609812:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973541 is committed 00000100:00100000:1.0:1789760765.612541:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760765.612632:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495879040 00000100:00100000:2.0:1789760765.612642:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495879040 00000100:00100000:2.0:1789760765.612693:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749860639155 time 749896658890 state 0x0 ref 1 00000100:00100000:2.0:1789760765.612708:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749896919306 time 749896658890 state 0x1 ref 1 00000100:00100000:2.0:1789760765.612738:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749896919306 time 749933455167 state 0x0 ref 1 00000100:00100000:2.0:1789760765.612750:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495879040:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760765.612797:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25497600, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc809 00010000:00010000:2.0:1789760765.612806:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547203 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.612822:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760765.612826:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.612833:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547203 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.612856:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547203 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760765.649828:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25497600 -> 25501696 00010000:00010000:2.0:1789760765.650010:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547203 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.650030:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547203 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.650039:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547203 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.650047:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760765.650051:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547203 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.650060:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547203 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.650082:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547203 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.650090:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547203 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760765.650180:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495879040:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37426us (37642us total) trans 4294973542 rc 0/0 00000100:00100000:2.0:1789760765.650190:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6254 00000100:00100000:2.0:1789760765.650201:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749896919306 time 749933455167 state 0x2 ref 0 00000001:00080000:2.0:1789760765.650243:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973542 is committed 00000100:00100000:1.0:1789760765.656462:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760765.656541:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495879168 00000100:00100000:3.0:1789760765.656550:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495879168 00000100:00100000:3.0:1789760765.656602:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749896919306 time 749933455167 state 0x0 ref 1 00000100:00100000:3.0:1789760765.656621:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749933715583 time 749933455167 state 0x1 ref 1 00000100:00100000:3.0:1789760765.656652:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749933715583 time 749977369053 state 0x0 ref 1 00000100:00100000:3.0:1789760765.656663:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495879168:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760765.656708:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25501696, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc809 00010000:00010000:3.0:1789760765.656716:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454720a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.656732:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760765.656736:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.656741:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454720a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.656754:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454720a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760765.678460:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973543 is committed 00002000:00010000:3.0:1789760765.678682:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25501696 -> 25505792 00002000:00010000:3.0:1789760765.678687:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760764 -> 1789760765 00002000:00010000:3.0:1789760765.678690:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760764 -> 1789760765 00010000:00010000:3.0:1789760765.678703:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454720a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.678716:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454720a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.678731:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454720a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.678740:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760765.678745:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454720a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.678754:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454720a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.678775:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454720a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.678783:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454720a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.679038:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495879168:12345-192.168.203.9@tcp:4:dd.0 Request processed in 22370us (22581us total) trans 4294973543 rc 0/0 00000100:00100000:3.0:1789760765.679048:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6255 00000100:00100000:3.0:1789760765.679059:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749933715583 time 749977369053 state 0x2 ref 0 00000100:00100000:3.0:1789760765.682718:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760765.682827:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495879296 00000100:00100000:0.0:1789760765.682837:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495879296 00000100:00100000:0.0:1789760765.682901:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749933715583 time 749977369053 state 0x0 ref 1 00000100:00100000:0.0:1789760765.682915:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749977629469 time 749977369053 state 0x1 ref 1 00000100:00100000:0.0:1789760765.682947:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749977629469 time 750003664326 state 0x0 ref 1 00000100:00100000:0.0:1789760765.682958:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495879296:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760765.683008:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25505792, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc809 00010000:00010000:0.0:1789760765.683017:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547211 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.683033:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760765.683037:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760765.683043:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547211 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.683059:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547211 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760765.721271:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25505792 -> 25509888 00010000:00010000:2.0:1789760765.721356:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547211 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760765.721523:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973544 is committed 00010000:00010000:2.0:1789760765.724046:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547211 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.724060:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547211 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.724070:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760765.724076:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547211 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.724087:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547211 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.724108:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547211 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.724115:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547211 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.724451:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495879296:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41489us (41738us total) trans 4294973544 rc 0/0 00000100:00100000:3.0:1789760765.724463:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6256 00000100:00100000:3.0:1789760765.724473:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749977629469 time 750003664326 state 0x2 ref 0 00000100:00100000:1.0:1789760765.728814:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760765.728902:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495879424 00000100:00100000:2.0:1789760765.728913:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495879424 00000100:00100000:2.0:1789760765.728967:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 749977629469 time 750003664326 state 0x0 ref 1 00000100:00100000:2.0:1789760765.728983:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750003924742 time 750003664326 state 0x1 ref 1 00000100:00100000:2.0:1789760765.729019:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750003924742 time 750049735748 state 0x0 ref 1 00000100:00100000:2.0:1789760765.729032:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495879424:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760765.729079:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25509888, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc809 00010000:00010000:2.0:1789760765.729088:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547218 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.729104:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:2.0:1789760765.729109:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.729115:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547218 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.729130:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547218 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760765.774504:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973545 is committed 00002000:00010000:2.0:1789760765.774599:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25509888 -> 25513984 00010000:00010000:2.0:1789760765.774616:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547218 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.774628:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547218 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.774638:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547218 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.774646:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760765.774651:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547218 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.774661:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547218 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.774680:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547218 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.774688:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547218 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760765.774817:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495879424:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45781us (46007us total) trans 4294973545 rc 0/0 00000100:00100000:2.0:1789760765.774827:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6257 00000100:00100000:2.0:1789760765.774837:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750003924742 time 750049735748 state 0x2 ref 0 00000100:00100000:1.0:1789760765.779570:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760765.779639:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495879552 00000100:00100000:3.0:1789760765.779649:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495879552 00000100:00100000:3.0:1789760765.779707:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750003924742 time 750049735748 state 0x0 ref 1 00000100:00100000:3.0:1789760765.779723:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750049996164 time 750049735748 state 0x1 ref 1 00000100:00100000:3.0:1789760765.779755:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750049996164 time 750100472406 state 0x0 ref 1 00000100:00100000:3.0:1789760765.779768:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495879552:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760765.779816:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25513984, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc809 00010000:00010000:3.0:1789760765.779825:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454721f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.780837:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760765.780851:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.780868:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454721f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.780887:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454721f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.790655:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760765.846787:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973546 is committed 00002000:00010000:2.0:1789760765.847098:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25513984 -> 25518080 00010000:00010000:2.0:1789760765.847117:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454721f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.847132:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454721f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.847141:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454721f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.847154:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760765.847159:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454721f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.847167:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454721f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.847256:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454721f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.847264:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454721f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.847543:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495879552:12345-192.168.203.9@tcp:4:dd.0 Request processed in 67771us (67977us total) trans 4294973546 rc 0/0 00000100:00100000:3.0:1789760765.847555:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6258 00000100:00100000:3.0:1789760765.847567:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750049996164 time 750100472406 state 0x2 ref 0 00000100:00100000:0.0:1789760765.850722:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760765.850823:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495879680 00000100:00100000:3.0:1789760765.850834:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495879680 00000100:00100000:3.0:1789760765.850901:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750049996164 time 750100472406 state 0x0 ref 1 00000100:00100000:3.0:1789760765.850918:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750100732822 time 750100472406 state 0x1 ref 1 00000100:00100000:3.0:1789760765.850947:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750100732822 time 750171663818 state 0x0 ref 1 00000100:00100000:3.0:1789760765.850960:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036e57100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495879680:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760765.851006:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25518080, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc809 00010000:00010000:3.0:1789760765.851014:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547226 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.851030:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760765.851034:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.851040:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547226 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.851053:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547226 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760765.894294:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973547 is committed 00002000:00010000:2.0:1789760765.894478:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25518080 -> 25522176 00010000:00010000:2.0:1789760765.894495:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547226 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.894508:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547226 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.894517:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547226 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.894524:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760765.894530:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547226 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.894541:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547226 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.894561:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547226 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.894568:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547226 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760765.894698:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036e57100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495879680:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43735us (43981us total) trans 4294973547 rc 0/0 00000100:00100000:2.0:1789760765.894710:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6259 00000100:00100000:2.0:1789760765.894721:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750100732822 time 750171663818 state 0x2 ref 0 00000100:00100000:0.0:1789760765.898528:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760765.901164:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495879808 00000100:00100000:2.0:1789760765.901191:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495879808 00000100:00100000:3.0:1789760765.901528:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750100732822 time 750171663818 state 0x0 ref 1 00000100:00100000:3.0:1789760765.901545:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750171924234 time 750171663818 state 0x1 ref 1 00000100:00100000:3.0:1789760765.901585:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750171924234 time 750222301713 state 0x0 ref 1 00000100:00100000:3.0:1789760765.901598:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036e55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495879808:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760765.901646:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25522176, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc809 00010000:00010000:3.0:1789760765.901655:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454722d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.901685:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760765.901690:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.901697:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454722d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.901710:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454722d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760765.944140:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973548 is committed 00002000:00010000:2.0:1789760765.944356:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25522176 -> 25526272 00010000:00010000:2.0:1789760765.944486:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454722d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.944499:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454722d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.944509:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454722d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.944522:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760765.944527:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454722d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.944536:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454722d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.944554:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454722d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.944562:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454722d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760765.944652:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036e55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495879808:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43052us (46129us total) trans 4294973548 rc 0/0 00000100:00100000:2.0:1789760765.944663:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6260 00000100:00100000:2.0:1789760765.944674:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750171924234 time 750222301713 state 0x2 ref 0 00000100:00100000:0.0:1789760765.952690:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760765.952777:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495879936 00000100:00100000:3.0:1789760765.952787:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495879936 00000100:00100000:3.0:1789760765.952854:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750171924234 time 750222301713 state 0x0 ref 1 00000100:00100000:3.0:1789760765.952881:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750222562129 time 750222301713 state 0x1 ref 1 00000100:00100000:3.0:1789760765.952910:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750222562129 time 750273626851 state 0x0 ref 1 00000100:00100000:3.0:1789760765.952923:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929018a80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495879936:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760765.952972:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25526272, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc809 00010000:00010000:3.0:1789760765.952980:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547234 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.953001:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760765.953007:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.953014:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547234 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.953027:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547234 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760766.032599:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973549 is committed 00002000:00010000:2.0:1789760766.034253:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25526272 -> 25530368 00010000:00010000:2.0:1789760766.034273:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547234 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.034302:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547234 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.034317:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547234 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.034325:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760766.034330:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547234 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.034340:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547234 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.034494:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547234 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.034505:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547234 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.034609:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929018a80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495879936:12345-192.168.203.9@tcp:4:dd.0 Request processed in 81684us (81926us total) trans 4294973549 rc 0/0 00000100:00100000:2.0:1789760766.034620:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6261 00000100:00100000:2.0:1789760766.034631:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750222562129 time 750273626851 state 0x2 ref 0 00000100:00100000:1.0:1789760766.040603:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760766.040693:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495880064 00000100:00100000:2.0:1789760766.040705:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495880064 00000100:00100000:2.0:1789760766.040763:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750222562129 time 750273626851 state 0x0 ref 1 00000100:00100000:2.0:1789760766.040779:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750273887267 time 750273626851 state 0x1 ref 1 00000100:00100000:2.0:1789760766.040810:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750273887267 time 750361527021 state 0x0 ref 1 00000100:00100000:2.0:1789760766.040823:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495880064:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760766.040981:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25530368, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc809 00010000:00010000:2.0:1789760766.040991:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454723b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.041009:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:2.0:1789760766.041014:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.041020:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454723b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.041035:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454723b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760766.096233:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973550 is committed 00002000:00010000:3.0:1789760766.096444:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25530368 -> 25534464 00010000:00010000:3.0:1789760766.096461:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454723b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.096475:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454723b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.096485:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454723b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.096492:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760766.096497:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454723b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.096507:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454723b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.096527:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454723b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.096536:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454723b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.096623:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495880064:12345-192.168.203.9@tcp:4:dd.0 Request processed in 55797us (56026us total) trans 4294973550 rc 0/0 00000100:00100000:3.0:1789760766.096633:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6262 00000100:00100000:3.0:1789760766.096643:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750273887267 time 750361527021 state 0x2 ref 0 00000100:00100000:0.0:1789760766.101016:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760766.102499:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495880192 00000100:00100000:2.0:1789760766.102513:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495880192 00000100:00100000:2.0:1789760766.102571:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750273887267 time 750361527021 state 0x0 ref 1 00000100:00100000:2.0:1789760766.102587:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750361787437 time 750361527021 state 0x1 ref 1 00000100:00100000:2.0:1789760766.102623:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750361787437 time 750423340649 state 0x0 ref 1 00000100:00100000:2.0:1789760766.102636:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495880192:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760766.102685:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25534464, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc809 00010000:00010000:2.0:1789760766.102695:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547242 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.102711:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:2.0:1789760766.102715:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.102722:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547242 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.102735:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547242 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760766.133217:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973551 is committed 00002000:00010000:2.0:1789760766.133947:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25534464 -> 25538560 00010000:00010000:2.0:1789760766.133967:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547242 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.133982:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547242 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.133992:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547242 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.134000:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760766.134005:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547242 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.134014:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547242 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.134040:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547242 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.134048:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547242 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.134189:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495880192:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31549us (33178us total) trans 4294973551 rc 0/0 00000100:00100000:2.0:1789760766.134200:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6263 00000100:00100000:2.0:1789760766.134209:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750361787437 time 750423340649 state 0x2 ref 0 00000100:00100000:0.0:1789760766.139143:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760766.139254:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495880320 00000100:00100000:2.0:1789760766.139263:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495880320 00000100:00100000:2.0:1789760766.139464:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750361787437 time 750423340649 state 0x0 ref 1 00000100:00100000:2.0:1789760766.139479:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750423601065 time 750423340649 state 0x1 ref 1 00000100:00100000:2.0:1789760766.139547:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750423601065 time 750460262954 state 0x0 ref 1 00000100:00100000:2.0:1789760766.139560:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495880320:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760766.139624:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25538560, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc809 00010000:00010000:2.0:1789760766.139636:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547249 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.139652:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:2.0:1789760766.139657:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.139663:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547249 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.139676:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547249 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.209771:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973552 is committed 00002000:00010000:0.0:1789760766.209928:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25538560 -> 25542656 00010000:00010000:0.0:1789760766.209944:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547249 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.209963:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547249 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.209973:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547249 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.209980:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760766.209985:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547249 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.209994:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547249 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.210016:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547249 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.210023:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547249 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.210156:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495880320:12345-192.168.203.9@tcp:4:dd.0 Request processed in 70594us (71017us total) trans 4294973552 rc 0/0 00000100:00100000:0.0:1789760766.210167:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6264 00000100:00100000:0.0:1789760766.210177:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750423601065 time 750460262954 state 0x2 ref 0 00000100:00100000:1.0:1789760766.215125:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760766.215430:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495880448 00000100:00100000:3.0:1789760766.215440:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495880448 00000100:00100000:3.0:1789760766.215491:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750423601065 time 750460262954 state 0x0 ref 1 00000100:00100000:3.0:1789760766.215506:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750460523370 time 750460262954 state 0x1 ref 1 00000100:00100000:3.0:1789760766.215564:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750460523370 time 750536280514 state 0x0 ref 1 00000100:00100000:3.0:1789760766.215577:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495880448:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760766.215620:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25542656, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc809 00010000:00010000:3.0:1789760766.215630:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547250 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.215645:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760766.215649:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.215656:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547250 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.215670:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547250 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.242731:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789760766.245529:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff929036f88a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520312064:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760766.245612:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760766.247053:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520312064 00000100:00100000:0.0:1789760766.247064:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520312064 00000100:00100000:0.0:1789760766.247132:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 209 00000100:00100000:0.0:1789760766.247142:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520312064:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789760766.247167:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527630336 pending 1703936 free 3832545280 avail 3798990848 00000100:00100000:0.0:1789760766.247231:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520312064:12345-0@lo:13:osp-pre-0-0.0 Request processed in 91us (1619us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760766.247238:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 209 00000100:00100000:0.0:1789760766.247309:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff929036f88a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520312064:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1789760766.247445:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789760766.251188:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25542656 -> 25546752 00010000:00010000:3.0:1789760766.251208:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547250 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.251220:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547250 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.251229:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547250 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.251236:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760766.251241:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547250 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.251249:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547250 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.251269:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547250 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.251276:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547250 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.251475:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495880448:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35896us (36355us total) trans 4294973553 rc 0/0 00000100:00100000:3.0:1789760766.251485:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6265 00000100:00100000:3.0:1789760766.251494:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750460523370 time 750536280514 state 0x2 ref 0 00000001:00080000:1.0:1789760766.252106:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973553 is committed 00000100:00100000:1.0:1789760766.258949:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760766.259077:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495880576 00000100:00100000:2.0:1789760766.259088:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495880576 00000100:00100000:2.0:1789760766.259142:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750460523370 time 750536280514 state 0x0 ref 1 00000100:00100000:2.0:1789760766.259158:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750536540930 time 750536280514 state 0x1 ref 1 00000100:00100000:2.0:1789760766.259189:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750536540930 time 750579905876 state 0x0 ref 1 00000100:00100000:2.0:1789760766.259200:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495880576:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760766.259249:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25546752, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc809 00010000:00010000:2.0:1789760766.259257:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547257 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.259276:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:2.0:1789760766.259323:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.259329:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547257 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.259345:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547257 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760766.292828:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25546752 -> 25550848 00010000:00010000:0.0:1789760766.292866:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547257 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.292881:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547257 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.292891:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547257 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.292898:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760766.292903:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547257 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.292912:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547257 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.292933:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547257 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.292945:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547257 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.293053:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495880576:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33849us (34107us total) trans 4294973554 rc 0/0 00000100:00100000:0.0:1789760766.293063:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6266 00000100:00100000:0.0:1789760766.293072:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750536540930 time 750579905876 state 0x2 ref 0 00000001:00080000:1.0:1789760766.293335:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973554 is committed 00000100:00100000:0.0:1789760766.297002:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760766.297728:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495880704 00000100:00100000:2.0:1789760766.297739:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495880704 00000100:00100000:2.0:1789760766.297795:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750536540930 time 750579905876 state 0x0 ref 1 00000100:00100000:2.0:1789760766.297811:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750580166292 time 750579905876 state 0x1 ref 1 00000100:00100000:2.0:1789760766.297843:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750580166292 time 750618560458 state 0x0 ref 1 00000100:00100000:2.0:1789760766.297865:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495880704:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760766.297913:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25550848, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc809 00010000:00010000:2.0:1789760766.297932:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454725e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.297949:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:2.0:1789760766.297954:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.297960:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454725e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.297974:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454725e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760766.335699:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973555 is committed 00002000:00010000:0.0:1789760766.335824:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25550848 -> 25554944 00010000:00010000:0.0:1789760766.335984:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454725e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.335998:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454725e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.336008:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454725e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.336015:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760766.336021:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454725e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.336030:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454725e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.336055:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454725e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.336063:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454725e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.336177:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495880704:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38318us (39179us total) trans 4294973555 rc 0/0 00000100:00100000:0.0:1789760766.336187:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6267 00000100:00100000:0.0:1789760766.336196:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750580166292 time 750618560458 state 0x2 ref 0 00000100:00100000:2.0:1789760766.342421:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760766.342547:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495880832 00000100:00100000:3.0:1789760766.342557:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495880832 00000100:00100000:3.0:1789760766.342615:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750580166292 time 750618560458 state 0x0 ref 1 00000100:00100000:3.0:1789760766.342632:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750618820874 time 750618560458 state 0x1 ref 1 00000100:00100000:3.0:1789760766.342665:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750618820874 time 750663382016 state 0x0 ref 1 00000100:00100000:3.0:1789760766.342677:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495880832:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760766.342726:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25554944, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc809 00010000:00010000:3.0:1789760766.342734:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547265 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.342752:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760766.342756:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.342763:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547265 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.342781:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547265 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760766.385483:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973556 is committed 00002000:00010000:0.0:1789760766.385830:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25554944 -> 25559040 00010000:00010000:0.0:1789760766.385847:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547265 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.385870:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547265 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.385879:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547265 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.385886:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760766.385890:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547265 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.385904:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547265 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.385925:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547265 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.385932:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547265 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.386032:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495880832:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43352us (43617us total) trans 4294973556 rc 0/0 00000100:00100000:0.0:1789760766.386043:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6268 00000100:00100000:0.0:1789760766.386054:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750618820874 time 750663382016 state 0x2 ref 0 00000100:00100000:1.0:1789760766.390054:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760766.390316:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495880960 00000100:00100000:0.0:1789760766.390324:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495880960 00000100:00100000:0.0:1789760766.390372:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750618820874 time 750663382016 state 0x0 ref 1 00000100:00100000:0.0:1789760766.390384:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750663642432 time 750663382016 state 0x1 ref 1 00000100:00100000:0.0:1789760766.390410:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750663642432 time 750711128474 state 0x0 ref 1 00000100:00100000:0.0:1789760766.390420:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495880960:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760766.390459:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25559040, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc809 00010000:00010000:0.0:1789760766.390465:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454726c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.390478:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760766.390482:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760766.390486:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454726c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.390495:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454726c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760766.423141:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973557 is committed 00002000:00010000:0.0:1789760766.423168:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25559040 -> 25563136 00010000:00010000:0.0:1789760766.423183:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454726c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.423201:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454726c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.423210:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454726c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.423218:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760766.423222:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454726c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.423233:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454726c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.423252:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454726c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.423260:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454726c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.423569:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495880960:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33146us (33519us total) trans 4294973557 rc 0/0 00000100:00100000:0.0:1789760766.423579:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6269 00000100:00100000:0.0:1789760766.423589:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750663642432 time 750711128474 state 0x2 ref 0 00000100:00100000:2.0:1789760766.427724:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760766.427774:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495881088 00000100:00100000:0.0:1789760766.427785:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495881088 00000100:00100000:0.0:1789760766.427863:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750663642432 time 750711128474 state 0x0 ref 1 00000100:00100000:0.0:1789760766.427879:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750711388890 time 750711128474 state 0x1 ref 1 00000100:00100000:0.0:1789760766.427908:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750711388890 time 750748625279 state 0x0 ref 1 00000100:00100000:0.0:1789760766.427921:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495881088:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760766.427965:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25563136, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc809 00010000:00010000:0.0:1789760766.427974:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547273 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.427991:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760766.427996:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760766.428002:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547273 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.428022:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547273 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760766.470349:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973558 is committed 00002000:00010000:3.0:1789760766.470462:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25563136 -> 25567232 00010000:00010000:3.0:1789760766.470477:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547273 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.470490:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547273 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.470498:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547273 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.470505:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760766.470510:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547273 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.470518:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547273 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.470537:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547273 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.470545:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547273 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.470650:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495881088:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42726us (42932us total) trans 4294973558 rc 0/0 00000100:00100000:3.0:1789760766.470661:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6270 00000100:00100000:3.0:1789760766.470670:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750711388890 time 750748625279 state 0x2 ref 0 00000100:00100000:1.0:1789760766.474556:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760766.474653:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495881216 00000100:00100000:0.0:1789760766.474663:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495881216 00000100:00100000:0.0:1789760766.474719:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750711388890 time 750748625279 state 0x0 ref 1 00000100:00100000:0.0:1789760766.474736:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750748885695 time 750748625279 state 0x1 ref 1 00000100:00100000:0.0:1789760766.474765:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750748885695 time 750795482652 state 0x0 ref 1 00000100:00100000:0.0:1789760766.474958:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495881216:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760766.475003:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25567232, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc809 00010000:00010000:0.0:1789760766.475011:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454727a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.475026:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760766.475031:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760766.475036:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454727a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.475050:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454727a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760766.517041:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25567232 -> 25571328 00010000:00010000:3.0:1789760766.517061:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454727a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.517074:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454727a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.517083:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454727a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.517091:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760766.517096:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454727a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.517104:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454727a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.517125:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454727a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.517139:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454727a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.517236:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495881216:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42276us (42685us total) trans 4294973559 rc 0/0 00000100:00100000:3.0:1789760766.517246:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6271 00000100:00100000:3.0:1789760766.517256:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750748885695 time 750795482652 state 0x2 ref 0 00000001:00080000:1.0:1789760766.518009:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973559 is committed 00000100:00100000:0.0:1789760766.522036:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760766.522344:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495881344 00000100:00100000:2.0:1789760766.522355:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495881344 00000100:00100000:3.0:1789760766.522479:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750748885695 time 750795482652 state 0x0 ref 1 00000100:00100000:3.0:1789760766.522495:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750795743068 time 750795482652 state 0x1 ref 1 00000100:00100000:3.0:1789760766.522529:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750795743068 time 750843245768 state 0x0 ref 1 00000100:00100000:3.0:1789760766.522542:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495881344:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760766.522587:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25571328, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc809 00010000:00010000:3.0:1789760766.522597:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547281 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.522613:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760766.522618:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.522626:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547281 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.522641:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547281 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760766.554184:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973560 is committed 00002000:00010000:3.0:1789760766.554235:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25571328 -> 25575424 00010000:00010000:3.0:1789760766.554250:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547281 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.554264:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547281 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.554273:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547281 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.554316:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760766.554321:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547281 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.554332:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547281 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.554351:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547281 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.554358:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547281 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.554443:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495881344:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31899us (32412us total) trans 4294973560 rc 0/0 00000100:00100000:3.0:1789760766.554457:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6272 00000100:00100000:3.0:1789760766.554466:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750795743068 time 750843245768 state 0x2 ref 0 00000100:00100000:0.0:1789760766.559264:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760766.559342:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495881472 00000100:00100000:3.0:1789760766.559351:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495881472 00000100:00100000:3.0:1789760766.559402:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750795743068 time 750843245768 state 0x0 ref 1 00000100:00100000:3.0:1789760766.559416:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750843506184 time 750843245768 state 0x1 ref 1 00000100:00100000:3.0:1789760766.559443:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750843506184 time 750880160486 state 0x0 ref 1 00000100:00100000:3.0:1789760766.559455:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495881472:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760766.559502:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25575424, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc809 00010000:00010000:3.0:1789760766.559510:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547288 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.559525:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760766.559529:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.559535:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547288 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.559549:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547288 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760766.605534:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973561 is committed 00002000:00010000:0.0:1789760766.605792:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25575424 -> 25579520 00010000:00010000:0.0:1789760766.605807:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547288 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.605821:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547288 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.605830:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547288 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.605839:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760766.605844:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547288 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.605861:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547288 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.605881:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547288 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.605889:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547288 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.606022:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495881472:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46564us (46762us total) trans 4294973561 rc 0/0 00000100:00100000:0.0:1789760766.606032:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6273 00000100:00100000:0.0:1789760766.606042:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750843506184 time 750880160486 state 0x2 ref 0 00000100:00100000:1.0:1789760766.610848:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760766.615148:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495881600 00000100:00100000:3.0:1789760766.615164:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495881600 00000100:00100000:3.0:1789760766.615230:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750843506184 time 750880160486 state 0x0 ref 1 00000100:00100000:3.0:1789760766.615251:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750880420902 time 750880160486 state 0x1 ref 1 00000100:00100000:3.0:1789760766.615302:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750880420902 time 750936018499 state 0x0 ref 1 00000100:00100000:3.0:1789760766.615318:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495881600:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760766.615371:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25579520, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc809 00010000:00010000:3.0:1789760766.615386:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454728f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.615403:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760766.615408:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.615416:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454728f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.615430:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454728f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760766.664642:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973562 is committed 00002000:00010000:3.0:1789760766.664862:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25579520 -> 25583616 00010000:00010000:3.0:1789760766.664879:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454728f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.664892:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454728f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.664902:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454728f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.664911:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760766.664916:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454728f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.664925:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454728f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.664944:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454728f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.664952:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454728f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.665050:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495881600:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49731us (54206us total) trans 4294973562 rc 0/0 00000100:00100000:3.0:1789760766.665062:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6274 00000100:00100000:3.0:1789760766.665072:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750880420902 time 750936018499 state 0x2 ref 0 00000100:00100000:0.0:1789760766.671375:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760766.671489:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495881728 00000100:00100000:3.0:1789760766.671499:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495881728 00000100:00100000:3.0:1789760766.671550:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750880420902 time 750936018499 state 0x0 ref 1 00000100:00100000:3.0:1789760766.671565:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750936278915 time 750936018499 state 0x1 ref 1 00000100:00100000:3.0:1789760766.671586:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750936278915 time 750992303101 state 0x0 ref 1 00000100:00100000:3.0:1789760766.671601:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495881728:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760766.671649:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25583616, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc809 00010000:00010000:3.0:1789760766.671658:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547296 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.671674:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760766.671678:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.671685:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547296 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.671699:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547296 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760766.702368:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973563 is committed 00002000:00010000:2.0:1789760766.702486:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25583616 -> 25587712 00002000:00010000:2.0:1789760766.702491:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760765 -> 1789760766 00002000:00010000:2.0:1789760766.702493:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760765 -> 1789760766 00010000:00010000:2.0:1789760766.702506:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547296 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.702518:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547296 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.702526:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547296 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.702533:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760766.702538:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547296 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.702545:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547296 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.702568:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547296 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.702576:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547296 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.702664:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495881728:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31061us (31294us total) trans 4294973563 rc 0/0 00000100:00100000:2.0:1789760766.702674:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6275 00000100:00100000:2.0:1789760766.702683:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750936278915 time 750992303101 state 0x2 ref 0 00000100:00100000:0.0:1789760766.707150:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760766.707231:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495881856 00000100:00100000:3.0:1789760766.707242:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495881856 00000100:00100000:3.0:1789760766.707327:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750936278915 time 750992303101 state 0x0 ref 1 00000100:00100000:3.0:1789760766.707344:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750992563517 time 750992303101 state 0x1 ref 1 00000100:00100000:3.0:1789760766.707372:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750992563517 time 751028089456 state 0x0 ref 1 00000100:00100000:2.0:1789760766.707441:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495881856:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760766.707492:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25587712, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc809 00010000:00010000:2.0:1789760766.707501:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454729d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.707517:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760766.707521:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.707528:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454729d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.707540:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454729d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760766.767163:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973564 is committed 00002000:00010000:2.0:1789760766.767386:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25587712 -> 25591808 00010000:00010000:2.0:1789760766.767403:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454729d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.767416:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454729d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.767425:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454729d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.767437:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760766.767442:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454729d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.767451:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454729d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.767471:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454729d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.767478:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454729d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.767565:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495881856:12345-192.168.203.9@tcp:4:dd.0 Request processed in 60179us (60427us total) trans 4294973564 rc 0/0 00000100:00100000:2.0:1789760766.767576:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6276 00000100:00100000:2.0:1789760766.767586:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750992563517 time 751028089456 state 0x2 ref 0 00000100:00100000:0.0:1789760766.778388:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760766.778495:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495881984 00000100:00100000:3.0:1789760766.778505:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495881984 00000100:00100000:3.0:1789760766.778561:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 750992563517 time 751028089456 state 0x0 ref 1 00000100:00100000:3.0:1789760766.778576:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751028349872 time 751028089456 state 0x1 ref 1 00000100:00100000:3.0:1789760766.778609:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751028349872 time 751099325622 state 0x0 ref 1 00000100:00100000:3.0:1789760766.778622:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495881984:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760766.778668:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25591808, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc809 00010000:00010000:3.0:1789760766.778677:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545472a4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.778692:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760766.778697:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.778704:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545472a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.778717:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545472a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760766.805615:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973565 is committed 00002000:00010000:2.0:1789760766.805794:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25591808 -> 25595904 00010000:00010000:2.0:1789760766.805815:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545472a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.805828:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545472a4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.805850:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545472a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.805865:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760766.805870:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545472a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.805879:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545472a4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.805898:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545472a4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.805905:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545472a4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.805998:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495881984:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27372us (27614us total) trans 4294973565 rc 0/0 00000100:00100000:2.0:1789760766.806009:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6277 00000100:00100000:2.0:1789760766.806019:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751028349872 time 751099325622 state 0x2 ref 0 00000100:00100000:0.0:1789760766.810013:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760766.810221:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495882112 00000100:00100000:2.0:1789760766.810231:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495882112 00000100:00100000:2.0:1789760766.810320:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751028349872 time 751099325622 state 0x0 ref 1 00000100:00100000:2.0:1789760766.810336:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751099586038 time 751099325622 state 0x1 ref 1 00000100:00100000:2.0:1789760766.810370:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751099586038 time 751131086911 state 0x0 ref 1 00000100:00100000:2.0:1789760766.810382:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495882112:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760766.810433:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25595904, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc809 00010000:00010000:2.0:1789760766.810442:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545472ab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.810459:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760766.810464:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.810471:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545472ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.810485:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545472ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.815370:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760766.844593:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973566 is committed 00002000:00010000:2.0:1789760766.844691:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25595904 -> 25600000 00010000:00010000:2.0:1789760766.844707:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545472ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.844720:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545472ab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.844730:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545472ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.844743:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760766.844749:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545472ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.844758:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545472ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.844779:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545472ab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.844788:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545472ab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.844931:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495882112:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34545us (34920us total) trans 4294973566 rc 0/0 00000100:00100000:2.0:1789760766.844941:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6278 00000100:00100000:2.0:1789760766.844950:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751099586038 time 751131086911 state 0x2 ref 0 00000100:00100000:0.0:1789760766.857869:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760766.857979:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495882240 00000100:00100000:2.0:1789760766.857988:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495882240 00000100:00100000:2.0:1789760766.858046:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751099586038 time 751131086911 state 0x0 ref 1 00000100:00100000:2.0:1789760766.858063:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751131347327 time 751131086911 state 0x1 ref 1 00000100:00100000:2.0:1789760766.858091:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751131347327 time 751178807348 state 0x0 ref 1 00000100:00100000:2.0:1789760766.858106:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495882240:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760766.858172:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25600000, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc809 00010000:00010000:2.0:1789760766.858189:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545472b2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.858205:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760766.858210:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.858217:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545472b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.858231:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545472b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760766.895971:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973567 is committed 00002000:00010000:3.0:1789760766.896064:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25600000 -> 25604096 00010000:00010000:3.0:1789760766.896081:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545472b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.896094:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545472b2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.896107:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545472b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.896117:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760766.896122:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545472b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.896131:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545472b2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.896151:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545472b2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.896158:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545472b2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.896255:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495882240:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38144us (38399us total) trans 4294973567 rc 0/0 00000100:00100000:3.0:1789760766.896265:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6279 00000100:00100000:3.0:1789760766.896274:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751131347327 time 751178807348 state 0x2 ref 0 00000100:00100000:1.0:1789760766.900500:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760766.900582:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495882368 00000100:00100000:2.0:1789760766.900592:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495882368 00000100:00100000:2.0:1789760766.900649:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751131347327 time 751178807348 state 0x0 ref 1 00000100:00100000:2.0:1789760766.900663:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751179067764 time 751178807348 state 0x1 ref 1 00000100:00100000:2.0:1789760766.900701:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751179067764 time 751221417379 state 0x0 ref 1 00000100:00100000:2.0:1789760766.900715:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f872680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495882368:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760766.900770:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25604096, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc809 00010000:00010000:2.0:1789760766.900779:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545472b9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.900806:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760766.900811:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.900817:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545472b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.900830:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545472b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.948317:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973568 is committed 00002000:00010000:1.0:1789760766.948463:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25604096 -> 25608192 00010000:00010000:1.0:1789760766.948479:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545472b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.948492:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545472b9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.948502:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545472b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.948509:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760766.948514:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545472b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.948522:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545472b9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.948552:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545472b9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.948560:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x49367483545472b9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.948648:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f872680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495882368:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47932us (48155us total) trans 4294973568 rc 0/0 00000100:00100000:1.0:1789760766.948659:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6280 00000100:00100000:1.0:1789760766.948668:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751179067764 time 751221417379 state 0x2 ref 0 00000100:00100000:0.0:1789760766.951653:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760766.951747:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495882496 00000100:00100000:2.0:1789760766.951770:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495882496 00000100:00100000:2.0:1789760766.951878:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751179067764 time 751221417379 state 0x0 ref 1 00000100:00100000:2.0:1789760766.951892:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751221677795 time 751221417379 state 0x1 ref 1 00000100:00100000:2.0:1789760766.951954:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751221677795 time 751272670075 state 0x0 ref 1 00000100:00100000:2.0:1789760766.951965:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495882496:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760766.952993:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25608192, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc809 00010000:00010000:2.0:1789760766.953007:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472c0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.953024:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760766.953034:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.953045:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.953065:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760766.990048:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973569 is committed 00002000:00010000:2.0:1789760766.990121:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25608192 -> 25612288 00010000:00010000:2.0:1789760766.990135:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.990146:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472c0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.990154:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.990161:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760766.990166:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.990173:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472c0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.990193:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472c0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.990200:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472c0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.990320:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495882496:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38351us (38672us total) trans 4294973569 rc 0/0 00000100:00100000:2.0:1789760766.990331:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6281 00000100:00100000:2.0:1789760766.990342:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751221677795 time 751272670075 state 0x2 ref 0 00000100:00100000:0.0:1789760766.998336:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760766.998433:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495882624 00000100:00100000:3.0:1789760766.998444:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495882624 00000100:00100000:3.0:1789760766.998513:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751221677795 time 751272670075 state 0x0 ref 1 00000100:00100000:3.0:1789760766.998528:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751272930491 time 751272670075 state 0x1 ref 1 00000100:00100000:3.0:1789760766.998559:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751272930491 time 751319275680 state 0x0 ref 1 00000100:00100000:3.0:1789760766.998572:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495882624:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760766.998623:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25612288, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc809 00010000:00010000:3.0:1789760766.998632:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472c7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.998649:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760766.998654:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.998661:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.998676:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760767.044243:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25612288 -> 25616384 00010000:00010000:3.0:1789760767.044264:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.044295:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472c7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.044306:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.044313:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760767.044319:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.044328:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472c7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.044349:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472c7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.044357:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472c7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.044453:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495882624:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45877us (46123us total) trans 4294973570 rc 0/0 00000100:00100000:3.0:1789760767.044463:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6282 00000100:00100000:3.0:1789760767.044474:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751272930491 time 751319275680 state 0x2 ref 0 00000001:00080000:2.0:1789760767.045085:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973570 is committed 00000100:00100000:2.0:1789760767.054683:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760767.055073:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495882752 00000100:00100000:1.0:1789760767.055083:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495882752 00000100:00100000:1.0:1789760767.055203:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751272930491 time 751319275680 state 0x0 ref 1 00000100:00100000:1.0:1789760767.055219:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751319536096 time 751319275680 state 0x1 ref 1 00000100:00100000:1.0:1789760767.055250:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751319536096 time 751375966750 state 0x0 ref 1 00000100:00100000:1.0:1789760767.055263:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034ce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495882752:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760767.055341:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25616384, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc809 00010000:00010000:1.0:1789760767.055350:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545472ce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.055367:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760767.055372:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760767.055379:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545472ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.055393:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545472ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760767.057303:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789760767.097886:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25616384 -> 25620480 00010000:00010000:1.0:1789760767.097908:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545472ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.097930:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545472ce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.097940:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545472ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.097946:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760767.097951:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545472ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.097959:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545472ce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.097979:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545472ce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.097989:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545472ce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.098085:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034ce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495882752:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42819us (43408us total) trans 4294973571 rc 0/0 00000100:00100000:1.0:1789760767.098096:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6283 00000100:00100000:1.0:1789760767.098106:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751319536096 time 751375966750 state 0x2 ref 0 00000001:00080000:2.0:1789760767.099416:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973571 is committed 00000100:00100000:1.0:1789760767.102226:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760767.102327:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495882880 00000100:00100000:2.0:1789760767.102337:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495882880 00000100:00100000:2.0:1789760767.102390:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751319536096 time 751375966750 state 0x0 ref 1 00000100:00100000:2.0:1789760767.102405:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751376227166 time 751375966750 state 0x1 ref 1 00000100:00100000:2.0:1789760767.102431:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751376227166 time 751423147953 state 0x0 ref 1 00000100:00100000:2.0:1789760767.102444:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495882880:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760767.102500:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25620480, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc809 00010000:00010000:2.0:1789760767.102509:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472d5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.102526:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760767.102531:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.102537:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.102556:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760767.131563:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973572 is committed 00002000:00010000:1.0:1789760767.132084:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25620480 -> 25624576 00010000:00010000:1.0:1789760767.132102:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.132122:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472d5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.132131:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.132140:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760767.132145:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.132155:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472d5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.132175:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472d5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.132182:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472d5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.132406:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495882880:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29960us (30185us total) trans 4294973572 rc 0/0 00000100:00100000:1.0:1789760767.132417:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6284 00000100:00100000:1.0:1789760767.132427:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751376227166 time 751423147953 state 0x2 ref 0 00000100:00100000:0.0:1789760767.141802:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760767.141866:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495883008 00000100:00100000:3.0:1789760767.141876:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495883008 00000100:00100000:3.0:1789760767.141928:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751376227166 time 751423147953 state 0x0 ref 1 00000100:00100000:3.0:1789760767.141982:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751423408369 time 751423147953 state 0x1 ref 1 00000100:00100000:3.0:1789760767.142016:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751423408369 time 751462731599 state 0x0 ref 1 00000100:00100000:3.0:1789760767.142028:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495883008:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760767.142075:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25624576, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc809 00010000:00010000:3.0:1789760767.142085:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472dc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.142117:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760767.142121:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.142128:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.142144:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760767.173924:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973573 is committed 00002000:00010000:3.0:1789760767.174070:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25624576 -> 25628672 00010000:00010000:3.0:1789760767.174085:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.174098:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472dc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.174107:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.174114:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760767.174120:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.174129:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472dc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.174151:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472dc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.174158:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472dc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.174323:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495883008:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32254us (32488us total) trans 4294973573 rc 0/0 00000100:00100000:3.0:1789760767.174334:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6285 00000100:00100000:3.0:1789760767.174345:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751423408369 time 751462731599 state 0x2 ref 0 00000100:00100000:2.0:1789760767.178917:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760767.179030:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495883136 00000100:00100000:1.0:1789760767.179041:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495883136 00000100:00100000:1.0:1789760767.179108:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751423408369 time 751462731599 state 0x0 ref 1 00000100:00100000:1.0:1789760767.179126:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751462992015 time 751462731599 state 0x1 ref 1 00000100:00100000:1.0:1789760767.179162:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751462992015 time 751499878039 state 0x0 ref 1 00000100:00100000:1.0:1789760767.179269:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495883136:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760767.179446:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25628672, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc809 00010000:00010000:1.0:1789760767.179455:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545472e3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.179473:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760767.179477:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760767.179555:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545472e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.179572:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545472e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.208234:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973574 is committed 00002000:00010000:0.0:1789760767.208529:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25628672 -> 25632768 00010000:00010000:0.0:1789760767.208546:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545472e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.208559:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545472e3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.208567:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545472e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.208578:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760767.208583:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545472e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.208592:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545472e3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.208612:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545472e3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.208619:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x49367483545472e3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.208713:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495883136:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29536us (29802us total) trans 4294973574 rc 0/0 00000100:00100000:0.0:1789760767.208724:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6286 00000100:00100000:0.0:1789760767.208734:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751462992015 time 751499878039 state 0x2 ref 0 00000100:00100000:1.0:1789760767.213664:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760767.213768:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495883264 00000100:00100000:2.0:1789760767.213778:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495883264 00000100:00100000:2.0:1789760767.213843:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751462992015 time 751499878039 state 0x0 ref 1 00000100:00100000:2.0:1789760767.213874:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751500138455 time 751499878039 state 0x1 ref 1 00000100:00100000:2.0:1789760767.213937:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751500138455 time 751534653266 state 0x0 ref 1 00000100:00100000:2.0:1789760767.213950:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495883264:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760767.214008:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25632768, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc809 00010000:00010000:2.0:1789760767.214017:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472ea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.214035:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760767.214040:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.214046:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.214060:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760767.239763:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973575 is committed 00002000:00010000:2.0:1789760767.239880:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25632768 -> 25636864 00010000:00010000:2.0:1789760767.239895:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.239908:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472ea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.239917:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.239925:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760767.239930:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.239938:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472ea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.239959:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472ea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.239967:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472ea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.240179:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495883264:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26226us (26520us total) trans 4294973575 rc 0/0 00000100:00100000:2.0:1789760767.240193:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6287 00000100:00100000:2.0:1789760767.240202:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751500138455 time 751534653266 state 0x2 ref 0 00000100:00100000:0.0:1789760767.244235:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760767.244396:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495883392 00000100:00100000:3.0:1789760767.244406:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495883392 00000100:00100000:3.0:1789760767.244458:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751500138455 time 751534653266 state 0x0 ref 1 00000100:00100000:3.0:1789760767.244474:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751534913682 time 751534653266 state 0x1 ref 1 00000100:00100000:3.0:1789760767.244544:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751534913682 time 751565260440 state 0x0 ref 1 00000100:00100000:3.0:1789760767.244558:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495883392:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760767.244606:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25636864, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc809 00010000:00010000:3.0:1789760767.244615:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472f1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.244631:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760767.244636:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.244643:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.244657:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.272154:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973576 is committed 00002000:00010000:1.0:1789760767.272422:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25636864 -> 25640960 00010000:00010000:1.0:1789760767.272438:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.272456:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472f1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.272465:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.272471:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760767.272475:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.272482:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472f1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.272501:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472f1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.272508:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545472f1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.272604:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495883392:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28042us (28434us total) trans 4294973576 rc 0/0 00000100:00100000:1.0:1789760767.272615:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6288 00000100:00100000:1.0:1789760767.272627:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751534913682 time 751565260440 state 0x2 ref 0 00000100:00100000:1.0:1789760767.275636:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760767.275966:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495883520 00000100:00100000:0.0:1789760767.275976:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495883520 00000100:00100000:0.0:1789760767.276028:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751534913682 time 751565260440 state 0x0 ref 1 00000100:00100000:0.0:1789760767.276043:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751565520856 time 751565260440 state 0x1 ref 1 00000100:00100000:0.0:1789760767.276075:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751565520856 time 751596792003 state 0x0 ref 1 00000100:00100000:0.0:1789760767.276088:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495883520:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760767.276135:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25640960, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc809 00010000:00010000:0.0:1789760767.276144:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545472f8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.276159:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:0.0:1789760767.276163:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.276169:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545472f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.276183:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545472f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760767.305408:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973577 is committed 00002000:00010000:1.0:1789760767.305777:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25640960 -> 25645056 00010000:00010000:1.0:1789760767.305794:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545472f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.305809:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545472f8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.305819:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545472f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.305897:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ae00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760767.305903:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545472f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.305912:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545472f8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.305933:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545472f8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.305941:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545472f8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.306072:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495883520:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29982us (30441us total) trans 4294973577 rc 0/0 00000100:00100000:1.0:1789760767.306082:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6289 00000100:00100000:1.0:1789760767.306091:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751565520856 time 751596792003 state 0x2 ref 0 00000100:00100000:0.0:1789760767.308795:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760767.309001:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495883648 00000100:00100000:2.0:1789760767.309011:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495883648 00000100:00100000:2.0:1789760767.309064:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751565520856 time 751596792003 state 0x0 ref 1 00000100:00100000:2.0:1789760767.309080:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751597052419 time 751596792003 state 0x1 ref 1 00000100:00100000:2.0:1789760767.309114:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751597052419 time 751629830619 state 0x0 ref 1 00000100:00100000:2.0:1789760767.309128:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495883648:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760767.309177:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25645056, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc809 00010000:00010000:2.0:1789760767.309187:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472ff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.309203:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760767.309208:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.309215:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.309228:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760767.336544:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973578 is committed 00002000:00010000:3.0:1789760767.336717:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25645056 -> 25649152 00010000:00010000:3.0:1789760767.336733:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.336745:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472ff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.336754:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.336761:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760767.336766:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.336775:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472ff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.336797:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472ff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.336805:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x49367483545472ff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.336956:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495883648:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27825us (28165us total) trans 4294973578 rc 0/0 00000100:00100000:3.0:1789760767.336966:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6290 00000100:00100000:3.0:1789760767.336976:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751597052419 time 751629830619 state 0x2 ref 0 00000100:00100000:0.0:1789760767.341021:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760767.341495:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495883776 00000100:00100000:1.0:1789760767.341505:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495883776 00000100:00100000:1.0:1789760767.341562:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751597052419 time 751629830619 state 0x0 ref 1 00000100:00100000:1.0:1789760767.341577:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751630091035 time 751629830619 state 0x1 ref 1 00000100:00100000:1.0:1789760767.341607:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751630091035 time 751662323800 state 0x0 ref 1 00000100:00100000:1.0:1789760767.341620:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495883776:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760767.341665:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25649152, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc809 00010000:00010000:1.0:1789760767.341674:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354547306 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.341689:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760767.341693:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760767.341699:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354547306 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.341713:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354547306 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760767.379452:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973579 is committed 00002000:00010000:3.0:1789760767.379566:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25649152 -> 25653248 00010000:00010000:3.0:1789760767.379580:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354547306 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.379592:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354547306 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.379601:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354547306 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.379609:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760767.379615:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354547306 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.379623:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354547306 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.379643:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354547306 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.379654:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354547306 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.379752:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495883776:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38127us (38733us total) trans 4294973579 rc 0/0 00000100:00100000:3.0:1789760767.379763:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6291 00000100:00100000:3.0:1789760767.379773:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751630091035 time 751662323800 state 0x2 ref 0 00000100:00100000:1.0:1789760767.388956:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760767.389085:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495883904 00000100:00100000:0.0:1789760767.389096:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495883904 00000100:00100000:0.0:1789760767.389164:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751630091035 time 751662323800 state 0x0 ref 1 00000100:00100000:0.0:1789760767.389179:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751662584216 time 751662323800 state 0x1 ref 1 00000100:00100000:0.0:1789760767.389207:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751662584216 time 751709924547 state 0x0 ref 1 00000100:00100000:0.0:1789760767.389220:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495883904:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760767.389499:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25653248, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc809 00010000:00010000:0.0:1789760767.389510:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454730d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.389527:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:0.0:1789760767.389531:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.389538:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454730d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.389553:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454730d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760767.420007:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973580 is committed 00002000:00010000:1.0:1789760767.420114:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25653248 -> 25657344 00010000:00010000:1.0:1789760767.420198:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454730d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.420212:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454730d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.420221:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454730d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.420229:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94aa00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760767.420234:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454730d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.420243:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454730d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.420264:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454730d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.420271:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454730d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.420458:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495883904:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31235us (31511us total) trans 4294973580 rc 0/0 00000100:00100000:1.0:1789760767.420468:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6292 00000100:00100000:1.0:1789760767.420478:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751662584216 time 751709924547 state 0x2 ref 0 00000100:00100000:1.0:1789760767.425001:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760767.425111:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495884032 00000100:00100000:3.0:1789760767.425193:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495884032 00000100:00100000:3.0:1789760767.425248:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751662584216 time 751709924547 state 0x0 ref 1 00000100:00100000:3.0:1789760767.425264:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751710184963 time 751709924547 state 0x1 ref 1 00000100:00100000:3.0:1789760767.425428:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751710184963 time 751746144115 state 0x0 ref 1 00000100:00100000:3.0:1789760767.425442:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495884032:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760767.425489:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25657344, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc809 00010000:00010000:3.0:1789760767.425498:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547314 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.425513:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760767.425518:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.425525:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547314 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.425544:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547314 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760767.459048:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973581 is committed 00002000:00010000:0.0:1789760767.459210:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25657344 -> 25661440 00010000:00010000:0.0:1789760767.459226:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547314 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.459239:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547314 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.459247:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547314 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.459254:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760767.459259:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547314 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.459267:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547314 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.459341:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547314 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.459349:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547314 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.459468:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495884032:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34024us (34473us total) trans 4294973581 rc 0/0 00000100:00100000:0.0:1789760767.459479:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6293 00000100:00100000:0.0:1789760767.459489:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751710184963 time 751746144115 state 0x2 ref 0 00000100:00100000:0.0:1789760767.462054:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760767.462267:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495884160 00000100:00100000:3.0:1789760767.462447:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495884160 00000100:00100000:2.0:1789760767.462577:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751710184963 time 751746144115 state 0x0 ref 1 00000100:00100000:2.0:1789760767.462595:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751746404531 time 751746144115 state 0x1 ref 1 00000100:00100000:2.0:1789760767.462662:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751746404531 time 751783378327 state 0x0 ref 1 00000100:00100000:2.0:1789760767.462676:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495884160:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760767.462726:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25661440, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc809 00010000:00010000:2.0:1789760767.462735:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454731b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.462751:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760767.462755:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.462761:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454731b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.462773:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454731b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.503338:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973582 is committed 00002000:00010000:2.0:1789760767.503362:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25661440 -> 25665536 00010000:00010000:2.0:1789760767.503377:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454731b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.503391:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454731b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.503400:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454731b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.503413:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760767.503418:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454731b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.503427:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454731b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.503445:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454731b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.503453:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454731b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.503556:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495884160:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40878us (41506us total) trans 4294973582 rc 0/0 00000100:00100000:2.0:1789760767.503567:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6294 00000100:00100000:2.0:1789760767.503576:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751746404531 time 751783378327 state 0x2 ref 0 00000100:00100000:1.0:1789760767.515514:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760767.515610:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495884288 00000100:00100000:2.0:1789760767.515620:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495884288 00000100:00100000:2.0:1789760767.515677:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751746404531 time 751783378327 state 0x0 ref 1 00000100:00100000:2.0:1789760767.515695:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751783638743 time 751783378327 state 0x1 ref 1 00000100:00100000:2.0:1789760767.515731:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751783638743 time 751836448400 state 0x0 ref 1 00000100:00100000:2.0:1789760767.515747:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036399850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495884288:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760767.515796:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25665536, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc809 00010000:00010000:2.0:1789760767.515805:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547322 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.515821:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760767.515825:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.515831:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547322 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.515843:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547322 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.559931:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973583 is committed 00002000:00010000:1.0:1789760767.560111:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25665536 -> 25669632 00010000:00010000:1.0:1789760767.560127:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547322 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.560141:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547322 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.560150:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547322 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.560158:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760767.560163:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547322 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.560174:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547322 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.560194:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547322 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.560201:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547322 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.560348:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036399850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495884288:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44597us (44841us total) trans 4294973583 rc 0/0 00000100:00100000:1.0:1789760767.560363:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6295 00000100:00100000:1.0:1789760767.560374:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751783638743 time 751836448400 state 0x2 ref 0 00000100:00100000:0.0:1789760767.577808:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760767.577927:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495884416 00000100:00100000:3.0:1789760767.577937:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495884416 00000100:00100000:3.0:1789760767.577989:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751783638743 time 751836448400 state 0x0 ref 1 00000100:00100000:3.0:1789760767.578003:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751836708816 time 751836448400 state 0x1 ref 1 00000100:00100000:3.0:1789760767.578036:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751836708816 time 751898753478 state 0x0 ref 1 00000100:00100000:3.0:1789760767.578049:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495884416:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760767.578100:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25669632, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc809 00010000:00010000:3.0:1789760767.578110:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547329 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.578125:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760767.578129:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.578136:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547329 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.578149:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547329 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760767.648831:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973584 is committed 00002000:00010000:3.0:1789760767.649460:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25669632 -> 25673728 00010000:00010000:3.0:1789760767.649478:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547329 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.649491:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547329 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.649501:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547329 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.649509:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760767.649514:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547329 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.649523:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547329 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.649543:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547329 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.649552:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547329 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.649645:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495884416:12345-192.168.203.9@tcp:4:dd.0 Request processed in 71593us (71846us total) trans 4294973584 rc 0/0 00000100:00100000:3.0:1789760767.649660:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6296 00000100:00100000:3.0:1789760767.649670:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751836708816 time 751898753478 state 0x2 ref 0 00000100:00100000:1.0:1789760767.655657:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760767.655824:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495884544 00000100:00100000:2.0:1789760767.655836:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495884544 00000100:00100000:2.0:1789760767.655904:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751836708816 time 751898753478 state 0x0 ref 1 00000100:00100000:2.0:1789760767.655920:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751899013894 time 751898753478 state 0x1 ref 1 00000100:00100000:2.0:1789760767.655954:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751899013894 time 751976670616 state 0x0 ref 1 00000100:00100000:2.0:1789760767.655967:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495884544:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760767.656021:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25673728, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc809 00010000:00010000:2.0:1789760767.656030:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547330 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.656051:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760767.656055:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.656061:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547330 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.656074:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547330 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760767.707505:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973585 is committed 00002000:00010000:3.0:1789760767.707582:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25673728 -> 25677824 00002000:00010000:3.0:1789760767.707588:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760766 -> 1789760767 00002000:00010000:3.0:1789760767.707592:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760766 -> 1789760767 00010000:00010000:3.0:1789760767.707606:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547330 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.707620:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547330 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.707628:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547330 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.707636:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760767.707641:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547330 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.707650:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547330 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.707673:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547330 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.707681:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547330 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.707774:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495884544:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51804us (52120us total) trans 4294973585 rc 0/0 00000100:00100000:3.0:1789760767.707785:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6297 00000100:00100000:3.0:1789760767.707795:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751899013894 time 751976670616 state 0x2 ref 0 00000100:00100000:1.0:1789760767.717372:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760767.719673:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495884672 00000100:00100000:2.0:1789760767.719687:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495884672 00000100:00100000:2.0:1789760767.719770:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751899013894 time 751976670616 state 0x0 ref 1 00000100:00100000:2.0:1789760767.719786:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751976931032 time 751976670616 state 0x1 ref 1 00000100:00100000:2.0:1789760767.719817:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751976931032 time 752040533863 state 0x0 ref 1 00000100:00100000:2.0:1789760767.719841:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495884672:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760767.719903:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25677824, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc809 00010000:00010000:2.0:1789760767.719912:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547337 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.719929:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:2.0:1789760767.719933:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.719939:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547337 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.719958:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547337 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760767.780112:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973586 is committed 00002000:00010000:3.0:1789760767.780208:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25677824 -> 25681920 00010000:00010000:3.0:1789760767.780227:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547337 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.780244:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547337 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.780257:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547337 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.780265:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760767.780270:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547337 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.780311:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547337 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.780335:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547337 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.780348:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547337 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.780452:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495884672:12345-192.168.203.9@tcp:4:dd.0 Request processed in 60618us (63086us total) trans 4294973586 rc 0/0 00000100:00100000:3.0:1789760767.780463:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6298 00000100:00100000:3.0:1789760767.780473:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751976931032 time 752040533863 state 0x2 ref 0 00000100:00100000:1.0:1789760767.785706:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760767.785784:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495884800 00000100:00100000:3.0:1789760767.785795:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495884800 00000100:00100000:3.0:1789760767.785871:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 751976931032 time 752040533863 state 0x0 ref 1 00000100:00100000:3.0:1789760767.785888:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752040794279 time 752040533863 state 0x1 ref 1 00000100:00100000:3.0:1789760767.785914:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752040794279 time 752106631600 state 0x0 ref 1 00000100:00100000:3.0:1789760767.785927:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495884800:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760767.785975:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25681920, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc809 00010000:00010000:3.0:1789760767.785985:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454733e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.786001:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760767.786006:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.786013:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454733e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.786034:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454733e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760767.824189:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25681920 -> 25686016 00010000:00010000:3.0:1789760767.824210:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454733e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.824226:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454733e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.824236:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454733e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.824243:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760767.824248:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454733e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.824258:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454733e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.824307:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454733e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.824316:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454733e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.824428:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495884800:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38496us (38727us total) trans 4294973587 rc 0/0 00000100:00100000:3.0:1789760767.824442:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6299 00000100:00100000:3.0:1789760767.824453:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752040794279 time 752106631600 state 0x2 ref 0 00000001:00080000:1.0:1789760767.824612:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973587 is committed 00000100:00100000:1.0:1789760767.831244:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760767.831349:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495884928 00000100:00100000:3.0:1789760767.831360:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495884928 00000100:00100000:3.0:1789760767.831415:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752040794279 time 752106631600 state 0x0 ref 1 00000100:00100000:3.0:1789760767.831431:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752106892016 time 752106631600 state 0x1 ref 1 00000100:00100000:3.0:1789760767.831461:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752106892016 time 752152177252 state 0x0 ref 1 00000100:00100000:3.0:1789760767.831474:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495884928:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760767.831526:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25686016, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc809 00010000:00010000:3.0:1789760767.831536:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547345 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.831553:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760767.831558:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.831565:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547345 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.831580:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547345 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.839867:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760767.886994:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25686016 -> 25690112 00010000:00010000:3.0:1789760767.887013:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547345 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.887026:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547345 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.887034:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547345 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.887041:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760767.887045:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547345 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.887054:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547345 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.887074:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547345 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.887081:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547345 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.887245:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495884928:12345-192.168.203.9@tcp:4:dd.0 Request processed in 55768us (56007us total) trans 4294973588 rc 0/0 00000100:00100000:3.0:1789760767.887260:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6300 00000100:00100000:3.0:1789760767.887270:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752106892016 time 752152177252 state 0x2 ref 0 00000001:00080000:1.0:1789760767.887432:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973588 is committed 00000100:00100000:2.0:1789760767.891492:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760767.891539:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495885056 00000100:00100000:3.0:1789760767.891550:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495885056 00000100:00100000:3.0:1789760767.891604:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752106892016 time 752152177252 state 0x0 ref 1 00000100:00100000:3.0:1789760767.891619:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752152437668 time 752152177252 state 0x1 ref 1 00000100:00100000:0.0:1789760767.891670:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752152437668 time 752212383847 state 0x0 ref 1 00000100:00100000:0.0:1789760767.891688:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495885056:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760767.891739:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25690112, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc809 00010000:00010000:0.0:1789760767.891748:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454734c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.891765:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760767.891770:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.891777:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454734c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.891797:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454734c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760767.936010:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973589 is committed 00002000:00010000:3.0:1789760767.936345:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25690112 -> 25694208 00010000:00010000:3.0:1789760767.936362:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454734c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.936375:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454734c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.936383:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454734c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.936390:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760767.936395:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454734c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.936512:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454734c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.936534:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454734c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.936543:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x493674835454734c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.936641:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495885056:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44954us (45154us total) trans 4294973589 rc 0/0 00000100:00100000:3.0:1789760767.936652:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6301 00000100:00100000:3.0:1789760767.936662:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752152437668 time 752212383847 state 0x2 ref 0 00000100:00100000:0.0:1789760767.940789:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760767.940861:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495885184 00000100:00100000:3.0:1789760767.940872:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495885184 00000100:00100000:3.0:1789760767.940924:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752152437668 time 752212383847 state 0x0 ref 1 00000100:00100000:3.0:1789760767.940940:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752212644263 time 752212383847 state 0x1 ref 1 00000100:00100000:3.0:1789760767.940973:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752212644263 time 752261689519 state 0x0 ref 1 00000100:00100000:3.0:1789760767.940986:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495885184:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760767.941037:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25694208, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc809 00010000:00010000:3.0:1789760767.941046:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547353 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.941061:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760767.941066:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.941073:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547353 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.941093:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547353 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.993706:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973590 is committed 00002000:00010000:0.0:1789760767.993809:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25694208 -> 25698304 00010000:00010000:0.0:1789760767.993838:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547353 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.993859:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547353 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.993868:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547353 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.993875:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760767.993881:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547353 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.993892:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547353 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.993911:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547353 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.993918:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547353 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.994031:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495885184:12345-192.168.203.9@tcp:4:dd.0 Request processed in 53040us (53244us total) trans 4294973590 rc 0/0 00000100:00100000:0.0:1789760767.994044:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6302 00000100:00100000:0.0:1789760767.994056:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752212644263 time 752261689519 state 0x2 ref 0 00000100:00100000:1.0:1789760767.998871:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760767.999117:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495885312 00000100:00100000:2.0:1789760767.999128:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495885312 00000100:00100000:2.0:1789760767.999197:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752212644263 time 752261689519 state 0x0 ref 1 00000100:00100000:2.0:1789760767.999212:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752261949935 time 752261689519 state 0x1 ref 1 00000100:00100000:2.0:1789760767.999245:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752261949935 time 752319961856 state 0x0 ref 1 00000100:00100000:2.0:1789760767.999258:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495885312:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760767.999330:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25698304, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc809 00010000:00010000:2.0:1789760767.999341:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454735a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.999358:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:2.0:1789760767.999363:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.999370:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454735a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.999383:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454735a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760768.055103:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973591 is committed 00002000:00010000:3.0:1789760768.055532:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25698304 -> 25702400 00010000:00010000:3.0:1789760768.055547:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454735a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.055560:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454735a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.055571:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454735a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.055579:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760768.055585:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454735a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.055594:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454735a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.055615:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454735a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.055623:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x493674835454735a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.055720:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495885312:12345-192.168.203.9@tcp:4:dd.0 Request processed in 56460us (56855us total) trans 4294973591 rc 0/0 00000100:00100000:3.0:1789760768.055731:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6303 00000100:00100000:3.0:1789760768.055740:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752261949935 time 752319961856 state 0x2 ref 0 00000100:00100000:1.0:1789760768.059641:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760768.059717:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495885440 00000100:00100000:3.0:1789760768.059728:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495885440 00000100:00100000:3.0:1789760768.059782:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752261949935 time 752319961856 state 0x0 ref 1 00000100:00100000:3.0:1789760768.059798:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752320222272 time 752319961856 state 0x1 ref 1 00000100:00100000:3.0:1789760768.059829:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752320222272 time 752380545349 state 0x0 ref 1 00000100:00100000:3.0:1789760768.059851:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495885440:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760768.059900:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25702400, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc809 00010000:00010000:3.0:1789760768.059909:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547361 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.059924:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760768.059929:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.059936:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547361 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.059950:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547361 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760768.106196:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25702400 -> 25706496 00010000:00010000:3.0:1789760768.106218:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547361 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.106231:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547361 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.106241:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547361 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.106248:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760768.106253:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547361 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.106276:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547361 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.106326:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547361 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.106334:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547361 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.106421:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495885440:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46577us (46785us total) trans 4294973592 rc 0/0 00000100:00100000:3.0:1789760768.106436:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6304 00000100:00100000:3.0:1789760768.106446:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752320222272 time 752380545349 state 0x2 ref 0 00000001:00080000:3.0:1789760768.106483:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973592 is committed 00000100:00100000:1.0:1789760768.114161:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760768.114257:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495885568 00000100:00100000:0.0:1789760768.114268:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495885568 00000100:00100000:0.0:1789760768.114359:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752320222272 time 752380545349 state 0x0 ref 1 00000100:00100000:0.0:1789760768.114375:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752380805765 time 752380545349 state 0x1 ref 1 00000100:00100000:0.0:1789760768.114410:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752380805765 time 752435126621 state 0x0 ref 1 00000100:00100000:0.0:1789760768.114423:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495885568:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760768.114466:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25706496, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc809 00010000:00010000:0.0:1789760768.114477:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547368 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.114493:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760768.114497:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.114504:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547368 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.114519:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547368 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.171545:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973593 is committed 00002000:00010000:3.0:1789760768.174008:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25706496 -> 25710592 00010000:00010000:3.0:1789760768.174028:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547368 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.174041:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547368 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.174050:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547368 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.174057:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760768.174061:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547368 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.174070:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547368 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.174090:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547368 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.174098:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547368 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.174529:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495885568:12345-192.168.203.9@tcp:4:dd.0 Request processed in 60102us (60370us total) trans 4294973593 rc 0/0 00000100:00100000:2.0:1789760768.174541:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6305 00000100:00100000:2.0:1789760768.174551:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752380805765 time 752435126621 state 0x2 ref 0 00000100:00100000:3.0:1789760768.182528:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760768.182684:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495885696 00000100:00100000:2.0:1789760768.182694:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495885696 00000100:00100000:2.0:1789760768.182747:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752380805765 time 752435126621 state 0x0 ref 1 00000100:00100000:2.0:1789760768.182764:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752435387037 time 752435126621 state 0x1 ref 1 00000100:00100000:2.0:1789760768.182795:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752435387037 time 752503511760 state 0x0 ref 1 00000100:00100000:2.0:1789760768.182808:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495885696:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760768.184412:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25710592, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc809 00010000:00010000:2.0:1789760768.184427:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454736f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.184445:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:2.0:1789760768.184450:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.184456:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454736f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.184470:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454736f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.235954:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973594 is committed 00002000:00010000:2.0:1789760768.236096:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25710592 -> 25714688 00010000:00010000:2.0:1789760768.236114:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454736f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.236127:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454736f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.236137:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454736f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.236145:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760768.236150:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454736f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.236168:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454736f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.236190:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454736f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.236198:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454736f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.236329:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495885696:12345-192.168.203.9@tcp:4:dd.0 Request processed in 53518us (53804us total) trans 4294973594 rc 0/0 00000100:00100000:2.0:1789760768.236342:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6306 00000100:00100000:2.0:1789760768.236352:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752435387037 time 752503511760 state 0x2 ref 0 00000100:00100000:1.0:1789760768.241156:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760768.241263:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495885824 00000100:00100000:3.0:1789760768.241422:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495885824 00000100:00100000:3.0:1789760768.241479:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752435387037 time 752503511760 state 0x0 ref 1 00000100:00100000:3.0:1789760768.241496:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752503772176 time 752503511760 state 0x1 ref 1 00000100:00100000:3.0:1789760768.241560:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752503772176 time 752562275912 state 0x0 ref 1 00000100:00100000:3.0:1789760768.241574:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495885824:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760768.241625:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25714688, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc809 00010000:00010000:3.0:1789760768.241634:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547376 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.241651:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760768.241657:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.241665:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547376 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.241680:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547376 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.272495:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973595 is committed 00002000:00010000:3.0:1789760768.272594:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25714688 -> 25718784 00010000:00010000:3.0:1789760768.272610:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547376 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.272629:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547376 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.272638:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547376 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.272646:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760768.272651:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547376 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.272660:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547376 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.272680:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547376 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.272688:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547376 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.272781:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495885824:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31205us (31629us total) trans 4294973595 rc 0/0 00000100:00100000:3.0:1789760768.272792:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6307 00000100:00100000:3.0:1789760768.272803:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752503772176 time 752562275912 state 0x2 ref 0 00000100:00100000:2.0:1789760768.276518:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760768.276672:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495885952 00000100:00100000:0.0:1789760768.276681:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495885952 00000100:00100000:0.0:1789760768.276729:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752503772176 time 752562275912 state 0x0 ref 1 00000100:00100000:0.0:1789760768.276743:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752562536328 time 752562275912 state 0x1 ref 1 00000100:00100000:0.0:1789760768.276773:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752562536328 time 752597489922 state 0x0 ref 1 00000100:00100000:0.0:1789760768.276784:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495885952:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760768.276935:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25718784, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc809 00010000:00010000:0.0:1789760768.276945:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454737d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.276962:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760768.276966:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.276973:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454737d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.276986:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454737d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.287446:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f02769f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:59c4e264-9b75-482b-8813-3d1dc124f384:3317:1876699520312192:0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.287517:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760768.287563:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f0276b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3317:1876699520312704:0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.287589:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760768.287614:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1789760768.287620:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f0276ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3317:1876699520312320:0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.287640:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760768.287665:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f02769c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3317:1876699520312832:0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.287687:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760768.287716:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f02768e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520312448:0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.287738:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760768.287757:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789760768.287762:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f02769880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520312576:0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.287782:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760768.287811:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520312192 00000100:00100000:3.0:1789760768.287817:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520312192 00000100:00100000:3.0:1789760768.287945:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 411 00000100:00100000:3.0:1789760768.287956:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929012313050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:59c4e264-9b75-482b-8813-3d1dc124f384+19:3317:x1876699520312192:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.288005:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929012313050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:59c4e264-9b75-482b-8813-3d1dc124f384+19:3317:x1876699520312192:12345-0@lo:400:kworker.0 Request processed in 51us (488us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760768.288014:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 411 00000100:00100000:3.0:1789760768.288046:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520312704 00000100:00100000:3.0:1789760768.288050:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520312704 00000100:00100000:3.0:1789760768.288081:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520312320 00000100:00100000:3.0:1789760768.288084:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520312320 00000100:00100000:3.0:1789760768.288101:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520312448 00000100:00100000:3.0:1789760768.288104:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520312448 00000100:00100000:3.0:1789760768.288119:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 507 00000100:00100000:3.0:1789760768.288126:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1876699520312704:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.288158:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1876699520312704:12345-0@lo:400:kworker.0 Request processed in 32us (568us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760768.288237:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 507 00000100:00100000:3.0:1789760768.288251:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 508 00000100:00100000:3.0:1789760768.288257:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1876699520312320:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.288872:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1876699520312320:12345-0@lo:400:kworker.0 Request processed in 613us (1228us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760768.289231:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 508 00000100:00100000:3.0:1789760768.289250:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 509 00000100:00100000:3.0:1789760768.289259:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd02d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520312448:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.289327:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd02d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520312448:12345-0@lo:400:kworker.0 Request processed in 69us (1587us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760768.289333:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 509 00000100:00100000:3.0:1789760768.289358:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520312832 00000100:00100000:3.0:1789760768.289363:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520312832 00000100:00100000:3.0:1789760768.289391:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520312576 00000100:00100000:3.0:1789760768.289405:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520312576 00000100:00100000:3.0:1789760768.289429:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 234 00000100:00100000:3.0:1789760768.289436:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd01c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3317:x1876699520312832:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.289468:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd01c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3317:x1876699520312832:12345-0@lo:400:kworker.0 Request processed in 33us (1780us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760768.289474:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 234 00000100:00100000:3.0:1789760768.289485:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 235 00000100:00100000:3.0:1789760768.289491:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520312576:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.289517:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520312576:12345-0@lo:400:kworker.0 Request processed in 26us (1734us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760768.289523:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 235 00000100:00080000:3.0:1789760768.289548:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789760768.289552:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789760768.289554:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760768.290296:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f02769880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520312576:0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.290312:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f02768e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520312448:0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.290352:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f0276ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3317:1876699520312320:0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.290362:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f02769c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3317:1876699520312832:0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.290371:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f02769f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:59c4e264-9b75-482b-8813-3d1dc124f384:3317:1876699520312192:0@lo:400:kworker.0 00000100:00100000:3.0:1789760768.290381:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f0276b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3317:1876699520312704:0@lo:400:kworker.0 00000001:00080000:3.0:1789760768.324012:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973596 is committed 00002000:00010000:0.0:1789760768.324086:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25718784 -> 25722880 00010000:00010000:0.0:1789760768.324102:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454737d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.324114:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454737d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.324123:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454737d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.324131:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760768.324136:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454737d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.324145:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454737d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.324164:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454737d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.324173:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454737d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.324265:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495885952:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47478us (47751us total) trans 4294973596 rc 0/0 00000100:00100000:0.0:1789760768.324275:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6308 00000100:00100000:0.0:1789760768.324332:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752562536328 time 752597489922 state 0x2 ref 0 00000100:00100000:1.0:1789760768.333615:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760768.333724:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495886080 00000100:00100000:2.0:1789760768.333734:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495886080 00000100:00100000:2.0:1789760768.333787:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752562536328 time 752597489922 state 0x0 ref 1 00000100:00100000:2.0:1789760768.333801:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752597750338 time 752597489922 state 0x1 ref 1 00000100:00100000:2.0:1789760768.333906:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752597750338 time 752654621471 state 0x0 ref 1 00000100:00100000:2.0:1789760768.333920:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495886080:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760768.333968:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25722880, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc809 00010000:00010000:2.0:1789760768.333978:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547384 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.333994:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:2.0:1789760768.333998:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.334004:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547384 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.334018:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547384 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760768.377374:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973597 is committed 00002000:00010000:2.0:1789760768.377618:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25722880 -> 25726976 00010000:00010000:2.0:1789760768.377634:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547384 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.377645:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547384 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.377654:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547384 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.377662:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760768.377667:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547384 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.377676:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547384 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.377695:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547384 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.377703:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547384 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.377793:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495886080:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43871us (44183us total) trans 4294973597 rc 0/0 00000100:00100000:2.0:1789760768.377803:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6309 00000100:00100000:2.0:1789760768.377812:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752597750338 time 752654621471 state 0x2 ref 0 00000100:00100000:1.0:1789760768.382119:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760768.382355:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495886208 00000100:00100000:3.0:1789760768.382366:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495886208 00000100:00100000:3.0:1789760768.382422:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752597750338 time 752654621471 state 0x0 ref 1 00000100:00100000:3.0:1789760768.382437:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752654881887 time 752654621471 state 0x1 ref 1 00000100:00100000:3.0:1789760768.382471:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752654881887 time 752703187480 state 0x0 ref 1 00000100:00100000:3.0:1789760768.382482:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495886208:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760768.382531:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25726976, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc809 00010000:00010000:3.0:1789760768.382540:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454738b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.382557:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760768.382562:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.382569:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454738b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.383002:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454738b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760768.428054:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973598 is committed 00002000:00010000:0.0:1789760768.428167:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25726976 -> 25731072 00010000:00010000:0.0:1789760768.428182:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454738b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.428195:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454738b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.428205:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454738b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.428214:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760768.428219:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454738b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.428229:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454738b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.428249:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454738b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.428256:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454738b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.428601:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495886208:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46116us (46487us total) trans 4294973598 rc 0/0 00000100:00100000:0.0:1789760768.428611:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6310 00000100:00100000:0.0:1789760768.428621:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752654881887 time 752703187480 state 0x2 ref 0 00000100:00100000:0.0:1789760768.431852:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760768.432111:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495886336 00000100:00100000:2.0:1789760768.432120:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495886336 00000100:00100000:2.0:1789760768.432243:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752654881887 time 752703187480 state 0x0 ref 1 00000100:00100000:2.0:1789760768.432259:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752703447896 time 752703187480 state 0x1 ref 1 00000100:00100000:2.0:1789760768.432324:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752703447896 time 752753039672 state 0x0 ref 1 00000100:00100000:2.0:1789760768.432338:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495886336:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760768.432565:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25731072, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc809 00010000:00010000:2.0:1789760768.432575:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547392 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.432590:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:2.0:1789760768.432595:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.432602:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547392 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.432615:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547392 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760768.467174:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973599 is committed 00002000:00010000:2.0:1789760768.467562:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25731072 -> 25735168 00010000:00010000:2.0:1789760768.467577:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547392 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.467597:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547392 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.467607:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547392 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.467615:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760768.467620:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547392 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.467629:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547392 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.467648:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547392 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.467655:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547392 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.467746:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495886336:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35405us (35915us total) trans 4294973599 rc 0/0 00000100:00100000:2.0:1789760768.467755:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6311 00000100:00100000:2.0:1789760768.467765:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752703447896 time 752753039672 state 0x2 ref 0 00000100:00100000:1.0:1789760768.473050:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760768.473143:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495886464 00000100:00100000:2.0:1789760768.473153:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495886464 00000100:00100000:2.0:1789760768.473273:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752703447896 time 752753039672 state 0x0 ref 1 00000100:00100000:2.0:1789760768.473306:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752753300088 time 752753039672 state 0x1 ref 1 00000100:00100000:2.0:1789760768.473432:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752753300088 time 752794147778 state 0x0 ref 1 00000100:00100000:2.0:1789760768.473445:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495886464:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760768.473496:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25735168, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc809 00010000:00010000:2.0:1789760768.473506:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547399 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.473524:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:2.0:1789760768.473528:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.473534:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547399 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.473549:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547399 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760768.508046:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973600 is committed 00002000:00010000:0.0:1789760768.508169:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25735168 -> 25739264 00010000:00010000:0.0:1789760768.508184:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547399 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.508198:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547399 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.508207:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547399 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.508215:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760768.508221:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547399 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.508231:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547399 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.508251:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547399 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.508260:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547399 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.508388:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495886464:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34940us (35343us total) trans 4294973600 rc 0/0 00000100:00100000:0.0:1789760768.508399:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6312 00000100:00100000:0.0:1789760768.508409:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752753300088 time 752794147778 state 0x2 ref 0 00000100:00100000:1.0:1789760768.512188:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760768.512269:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495886592 00000100:00100000:0.0:1789760768.512379:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495886592 00000100:00100000:0.0:1789760768.512443:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752753300088 time 752794147778 state 0x0 ref 1 00000100:00100000:0.0:1789760768.512459:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752794408194 time 752794147778 state 0x1 ref 1 00000100:00100000:0.0:1789760768.512539:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752794408194 time 752833255498 state 0x0 ref 1 00000100:00100000:0.0:1789760768.512553:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495886592:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760768.512603:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25739264, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc809 00010000:00010000:0.0:1789760768.512612:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473a0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.512627:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760768.512631:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.512637:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.512651:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760768.539461:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25739264 -> 25743360 00010000:00010000:0.0:1789760768.539482:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.539496:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473a0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.539506:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.539514:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760768.539519:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.539528:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473a0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.539551:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473a0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.539559:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473a0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.539658:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495886592:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27104us (27475us total) trans 4294973601 rc 0/0 00000100:00100000:0.0:1789760768.539669:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6313 00000100:00100000:0.0:1789760768.539679:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752794408194 time 752833255498 state 0x2 ref 0 00000001:00080000:2.0:1789760768.540132:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973601 is committed 00000100:00100000:2.0:1789760768.543598:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760768.543737:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495886720 00000100:00100000:0.0:1789760768.543747:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495886720 00000100:00100000:0.0:1789760768.543798:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752794408194 time 752833255498 state 0x0 ref 1 00000100:00100000:0.0:1789760768.543811:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752833515914 time 752833255498 state 0x1 ref 1 00000100:00100000:0.0:1789760768.543895:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752833515914 time 752864611103 state 0x0 ref 1 00000100:00100000:0.0:1789760768.543909:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495886720:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760768.543954:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25743360, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc809 00010000:00010000:0.0:1789760768.543962:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545473a7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.543977:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760768.543982:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.543987:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545473a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.544001:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545473a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760768.571012:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973602 is committed 00002000:00010000:0.0:1789760768.571103:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25743360 -> 25747456 00010000:00010000:0.0:1789760768.571119:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545473a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.571131:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545473a7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.571146:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545473a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.571153:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760768.571158:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545473a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.571167:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545473a7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.571185:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545473a7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.571192:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545473a7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.571339:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495886720:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27428us (27745us total) trans 4294973602 rc 0/0 00000100:00100000:0.0:1789760768.571349:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6314 00000100:00100000:0.0:1789760768.571359:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752833515914 time 752864611103 state 0x2 ref 0 00000100:00100000:3.0:1789760768.575174:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760768.577002:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495886848 00000100:00100000:0.0:1789760768.577017:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495886848 00000100:00100000:0.0:1789760768.577078:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752833515914 time 752864611103 state 0x0 ref 1 00000100:00100000:0.0:1789760768.577093:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752864871519 time 752864611103 state 0x1 ref 1 00000100:00100000:0.0:1789760768.577168:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752864871519 time 752897883465 state 0x0 ref 1 00000100:00100000:0.0:1789760768.577182:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495886848:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760768.577228:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25747456, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc809 00010000:00010000:0.0:1789760768.577237:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545473ae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.577255:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760768.577259:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.577266:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545473ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.577295:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545473ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760768.609469:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973603 is committed 00002000:00010000:0.0:1789760768.609553:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25747456 -> 25751552 00010000:00010000:0.0:1789760768.609569:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545473ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.609582:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545473ae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.609591:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545473ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.609599:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760768.609604:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545473ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.609612:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545473ae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.609630:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545473ae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.609637:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545473ae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.609761:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495886848:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32577us (34592us total) trans 4294973603 rc 0/0 00000100:00100000:0.0:1789760768.609772:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6315 00000100:00100000:0.0:1789760768.609782:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752864871519 time 752897883465 state 0x2 ref 0 00000100:00100000:1.0:1789760768.614576:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760768.614637:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495886976 00000100:00100000:0.0:1789760768.614647:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495886976 00000100:00100000:0.0:1789760768.614698:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752864871519 time 752897883465 state 0x0 ref 1 00000100:00100000:0.0:1789760768.614712:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752898143881 time 752897883465 state 0x1 ref 1 00000100:00100000:0.0:1789760768.614738:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752898143881 time 752935455482 state 0x0 ref 1 00000100:00100000:0.0:1789760768.614751:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495886976:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760768.614796:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25751552, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc809 00010000:00010000:0.0:1789760768.614806:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473b5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.614823:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760768.614827:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.614834:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.614858:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760768.660250:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973604 is committed 00002000:00010000:0.0:1789760768.660473:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25751552 -> 25755648 00010000:00010000:0.0:1789760768.660490:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.660503:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473b5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.660512:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.660520:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760768.660525:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.660533:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.660556:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473b5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.660564:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473b5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.660670:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495886976:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45916us (46098us total) trans 4294973604 rc 0/0 00000100:00100000:0.0:1789760768.660680:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6316 00000100:00100000:0.0:1789760768.660689:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752898143881 time 752935455482 state 0x2 ref 0 00000100:00100000:2.0:1789760768.672860:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760768.673324:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495887104 00000100:00100000:0.0:1789760768.673334:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495887104 00000100:00100000:0.0:1789760768.673387:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752898143881 time 752935455482 state 0x0 ref 1 00000100:00100000:0.0:1789760768.673403:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752935715898 time 752935455482 state 0x1 ref 1 00000100:00100000:0.0:1789760768.673434:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752935715898 time 752994150995 state 0x0 ref 1 00000100:00100000:0.0:1789760768.673446:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495887104:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760768.673495:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25755648, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc809 00010000:00010000:0.0:1789760768.673504:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473bc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.673519:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760768.673524:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.673531:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.673544:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.726056:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973605 is committed 00002000:00010000:0.0:1789760768.726130:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25755648 -> 25759744 00002000:00010000:0.0:1789760768.726134:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760767 -> 1789760768 00002000:00010000:0.0:1789760768.726137:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760767 -> 1789760768 00010000:00010000:0.0:1789760768.726150:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.726162:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473bc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.726172:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.726179:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760768.726184:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.726194:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473bc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.726218:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473bc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.726227:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473bc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.726354:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495887104:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52903us (53496us total) trans 4294973605 rc 0/0 00000100:00100000:0.0:1789760768.726363:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6317 00000100:00100000:0.0:1789760768.726374:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752935715898 time 752994150995 state 0x2 ref 0 00000100:00100000:2.0:1789760768.733945:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760768.734107:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495887232 00000100:00100000:0.0:1789760768.734116:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495887232 00000100:00100000:0.0:1789760768.734172:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752935715898 time 752994150995 state 0x0 ref 1 00000100:00100000:0.0:1789760768.734188:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752994411411 time 752994150995 state 0x1 ref 1 00000100:00100000:0.0:1789760768.734220:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752994411411 time 753054936450 state 0x0 ref 1 00000100:00100000:0.0:1789760768.734234:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495887232:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760768.734315:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25759744, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc809 00010000:00010000:0.0:1789760768.734324:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473c3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.734340:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760768.734345:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.734351:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.734371:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760768.775979:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25759744 -> 25763840 00010000:00010000:0.0:1789760768.775999:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.776013:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473c3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.776022:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.776029:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760768.776034:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.776043:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473c3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.776064:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473c3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.776071:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473c3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.776334:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495887232:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42049us (42345us total) trans 4294973606 rc 0/0 00000100:00100000:0.0:1789760768.776345:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6318 00000100:00100000:0.0:1789760768.776651:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752994411411 time 753054936450 state 0x2 ref 0 00000001:00080000:0.0:1789760768.776691:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973606 is committed 00000100:00100000:0.0:1789760768.786990:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760768.787276:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495887360 00000100:00100000:3.0:1789760768.787330:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495887360 00000100:00100000:3.0:1789760768.787383:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 752994411411 time 753054936450 state 0x0 ref 1 00000100:00100000:3.0:1789760768.787397:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753055196866 time 753054936450 state 0x1 ref 1 00000100:00100000:3.0:1789760768.787427:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753055196866 time 753108144620 state 0x0 ref 1 00000100:00100000:3.0:1789760768.787439:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495887360:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760768.787484:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25763840, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc809 00010000:00010000:3.0:1789760768.787492:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545473ca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.787508:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:3.0:1789760768.787513:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.787519:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545473ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.787533:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545473ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.798562:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff929005d87100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520312960:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760768.798632:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760768.798707:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520312960 00000100:00100000:0.0:1789760768.798715:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520312960 00000100:00100000:0.0:1789760768.798762:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 210 00000100:00100000:0.0:1789760768.798772:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520312960:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789760768.798809:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760768.798933:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520312960:12345-0@lo:13:osp-pre-1-0.0 Request processed in 163us (301us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760768.798941:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 210 00000100:00100000:0.0:1789760768.798989:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff929005d87100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520312960:0@lo:13:osp-pre-1-0.0 00000001:00080000:2.0:1789760768.824048:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973607 is committed 00002000:00010000:1.0:1789760768.824149:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25763840 -> 25767936 00010000:00010000:1.0:1789760768.824165:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545473ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.824178:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545473ca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.824187:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545473ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.824194:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760768.824199:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545473ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.824207:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545473ca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.824227:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545473ca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.824234:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545473ca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760768.824360:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495887360:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36918us (37374us total) trans 4294973607 rc 0/0 00000100:00100000:1.0:1789760768.824371:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6319 00000100:00100000:1.0:1789760768.824380:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753055196866 time 753108144620 state 0x2 ref 0 00000100:00100000:2.0:1789760768.832766:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760768.832868:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495887488 00000100:00100000:0.0:1789760768.832877:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495887488 00000100:00100000:0.0:1789760768.832928:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753055196866 time 753108144620 state 0x0 ref 1 00000100:00100000:0.0:1789760768.832943:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753108405036 time 753108144620 state 0x1 ref 1 00000100:00100000:0.0:1789760768.832975:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753108405036 time 753153690993 state 0x0 ref 1 00000100:00100000:0.0:1789760768.832988:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495887488:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760768.833039:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25767936, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc809 00010000:00010000:0.0:1789760768.833048:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473d1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.833065:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760768.833071:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.833078:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.833093:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.862724:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760768.870771:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973608 is committed 00002000:00010000:1.0:1789760768.870989:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25767936 -> 25772032 00010000:00010000:1.0:1789760768.871007:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.871020:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473d1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.872499:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.872516:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760768.872521:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.872532:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473d1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.872553:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473d1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.872561:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x49367483545473d1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760768.872665:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495887488:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39674us (39903us total) trans 4294973608 rc 0/0 00000100:00100000:1.0:1789760768.872676:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6320 00000100:00100000:1.0:1789760768.872687:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753108405036 time 753153690993 state 0x2 ref 0 00000100:00100000:3.0:1789760768.886419:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760768.886541:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495887616 00000100:00100000:0.0:1789760768.886552:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495887616 00000100:00100000:0.0:1789760768.886952:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753108405036 time 753153690993 state 0x0 ref 1 00000100:00100000:0.0:1789760768.887065:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753153951409 time 753153690993 state 0x1 ref 1 00000100:00100000:0.0:1789760768.887099:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753153951409 time 753207815095 state 0x0 ref 1 00000100:00100000:0.0:1789760768.887182:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495887616:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760768.887236:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25772032, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc809 00010000:00010000:0.0:1789760768.887246:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545473d8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.887267:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760768.887272:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.887828:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545473d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.888028:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545473d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760768.950730:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973609 is committed 00002000:00010000:2.0:1789760768.950863:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25772032 -> 25776128 00010000:00010000:2.0:1789760768.950879:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545473d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.950892:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545473d8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.950902:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545473d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.950909:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760768.950914:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545473d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.950923:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545473d8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.950942:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545473d8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.950950:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545473d8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.951052:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495887616:12345-192.168.203.9@tcp:4:dd.0 Request processed in 63938us (64639us total) trans 4294973609 rc 0/0 00000100:00100000:2.0:1789760768.951063:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6321 00000100:00100000:2.0:1789760768.951074:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753153951409 time 753207815095 state 0x2 ref 0 00000100:00100000:0.0:1789760768.955606:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760768.959323:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495887744 00000100:00100000:1.0:1789760768.959337:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495887744 00000100:00100000:1.0:1789760768.959399:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753153951409 time 753207815095 state 0x0 ref 1 00000100:00100000:1.0:1789760768.959415:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753208075511 time 753207815095 state 0x1 ref 1 00000100:00100000:1.0:1789760768.959448:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753208075511 time 753280165079 state 0x0 ref 1 00000100:00100000:1.0:1789760768.959463:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495887744:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760768.959511:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25776128, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc809 00010000:00010000:1.0:1789760768.959521:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545473df lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.959536:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760768.959541:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760768.959548:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545473df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.959563:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545473df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760768.998741:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973610 is committed 00002000:00010000:1.0:1789760768.998880:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25776128 -> 25780224 00010000:00010000:1.0:1789760768.998894:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545473df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.998914:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545473df lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.998923:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545473df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.998929:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760768.998935:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545473df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.998943:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545473df lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.998967:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545473df lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.998976:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545473df lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760768.999075:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495887744:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39611us (43484us total) trans 4294973610 rc 0/0 00000100:00100000:1.0:1789760768.999085:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6322 00000100:00100000:1.0:1789760768.999095:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753208075511 time 753280165079 state 0x2 ref 0 00000100:00100000:3.0:1789760769.002648:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760769.002945:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495887872 00000100:00100000:1.0:1789760769.002955:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495887872 00000100:00100000:1.0:1789760769.003022:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753208075511 time 753280165079 state 0x0 ref 1 00000100:00100000:1.0:1789760769.003038:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753280425495 time 753280165079 state 0x1 ref 1 00000100:00100000:1.0:1789760769.003059:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753280425495 time 753323776019 state 0x0 ref 1 00000100:00100000:1.0:1789760769.003070:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495887872:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760769.003122:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25780224, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc809 00010000:00010000:1.0:1789760769.003132:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473e6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.003148:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760769.003153:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00000100:00080000:3.0:1789760769.003301:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789760769.003400:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.003416:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.030211:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973611 is committed 00002000:00010000:1.0:1789760769.030386:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25780224 -> 25784320 00010000:00010000:1.0:1789760769.030401:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.030414:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473e6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.030429:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.030436:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760769.030441:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.030450:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473e6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.030468:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473e6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.030476:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473e6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.030552:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495887872:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27480us (27908us total) trans 4294973611 rc 0/0 00000100:00100000:1.0:1789760769.030561:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6323 00000100:00100000:1.0:1789760769.030570:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753280425495 time 753323776019 state 0x2 ref 0 00000100:00100000:3.0:1789760769.034733:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760769.035051:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495888000 00000100:00100000:0.0:1789760769.035059:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495888000 00000100:00100000:0.0:1789760769.035108:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753280425495 time 753323776019 state 0x0 ref 1 00000100:00100000:0.0:1789760769.035121:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753324036435 time 753323776019 state 0x1 ref 1 00000100:00100000:0.0:1789760769.035152:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753324036435 time 753355868827 state 0x0 ref 1 00000100:00100000:0.0:1789760769.035243:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495888000:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760769.035322:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25784320, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc809 00010000:00010000:0.0:1789760769.035330:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545473ed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.035346:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760769.035351:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.035357:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545473ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.035371:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545473ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760769.056159:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25784320 -> 25788416 00010000:00010000:0.0:1789760769.056180:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545473ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.056194:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545473ed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.056204:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545473ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.056212:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760769.056218:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545473ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.056227:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545473ed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.056246:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545473ed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.056255:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545473ed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.056387:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495888000:12345-192.168.203.9@tcp:4:dd.0 Request processed in 21220us (21657us total) trans 4294973612 rc 0/0 00000100:00100000:0.0:1789760769.056397:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6324 00000100:00100000:0.0:1789760769.056407:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753324036435 time 753355868827 state 0x2 ref 0 00000001:00080000:3.0:1789760769.056496:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973612 is committed 00000100:00100000:3.0:1789760769.059705:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760769.060141:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495888128 00000100:00100000:1.0:1789760769.060153:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495888128 00000100:00100000:1.0:1789760769.060223:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753324036435 time 753355868827 state 0x0 ref 1 00000100:00100000:1.0:1789760769.060237:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753356129243 time 753355868827 state 0x1 ref 1 00000100:00100000:1.0:1789760769.060271:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753356129243 time 753380988116 state 0x0 ref 1 00000100:00100000:1.0:1789760769.060314:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02768e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495888128:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760769.060365:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25788416, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc809 00010000:00010000:1.0:1789760769.060374:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473f4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.060390:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760769.060395:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.060401:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.060415:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.105794:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973613 is committed 00002000:00010000:1.0:1789760769.105993:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25788416 -> 25792512 00010000:00010000:1.0:1789760769.106008:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.106021:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473f4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.106030:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.106037:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760769.106041:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.106050:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473f4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.106069:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473f4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.106076:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473f4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.106166:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02768e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495888128:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45851us (46467us total) trans 4294973613 rc 0/0 00000100:00100000:1.0:1789760769.106176:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6325 00000100:00100000:1.0:1789760769.106186:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753356129243 time 753380988116 state 0x2 ref 0 00000100:00100000:3.0:1789760769.112724:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760769.112830:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495888256 00000100:00100000:1.0:1789760769.112849:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495888256 00000100:00100000:1.0:1789760769.112904:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753356129243 time 753380988116 state 0x0 ref 1 00000100:00100000:1.0:1789760769.112919:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753381248532 time 753380988116 state 0x1 ref 1 00000100:00100000:1.0:1789760769.112961:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753381248532 time 753433678035 state 0x0 ref 1 00000100:00100000:1.0:1789760769.112974:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495888256:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760769.113020:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25792512, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc809 00010000:00010000:1.0:1789760769.113030:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473fb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.113045:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760769.113050:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.113056:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.113070:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760769.146313:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25792512 -> 25796608 00010000:00010000:1.0:1789760769.146331:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.146344:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473fb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.146353:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.146360:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760769.146364:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.146372:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.146397:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473fb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.146404:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545473fb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.146492:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495888256:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33516us (33772us total) trans 4294973614 rc 0/0 00000100:00100000:1.0:1789760769.146502:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6326 00000100:00100000:1.0:1789760769.146511:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753381248532 time 753433678035 state 0x2 ref 0 00000001:00080000:3.0:1789760769.146950:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973614 is committed 00000100:00100000:3.0:1789760769.155736:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760769.155816:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495888384 00000100:00100000:0.0:1789760769.155826:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495888384 00000100:00100000:0.0:1789760769.155895:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753381248532 time 753433678035 state 0x0 ref 1 00000100:00100000:0.0:1789760769.155912:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753433938451 time 753433678035 state 0x1 ref 1 00000100:00100000:0.0:1789760769.155998:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753433938451 time 753476713723 state 0x0 ref 1 00000100:00100000:0.0:1789760769.156012:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495888384:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760769.156058:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25796608, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc809 00010000:00010000:0.0:1789760769.156068:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547402 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.156083:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760769.156263:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.156271:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547402 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.156329:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547402 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.221522:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973615 is committed 00002000:00010000:2.0:1789760769.221950:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25796608 -> 25800704 00010000:00010000:2.0:1789760769.221966:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547402 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.221979:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547402 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.221989:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547402 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.221997:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760769.222002:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547402 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.222011:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547402 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.222031:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547402 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.222038:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547402 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.222132:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495888384:12345-192.168.203.9@tcp:4:dd.0 Request processed in 66119us (66401us total) trans 4294973615 rc 0/0 00000100:00100000:2.0:1789760769.222142:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6327 00000100:00100000:2.0:1789760769.222153:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753433938451 time 753476713723 state 0x2 ref 0 00000100:00100000:0.0:1789760769.229913:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760769.230321:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495888512 00000100:00100000:1.0:1789760769.230331:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495888512 00000100:00100000:1.0:1789760769.230389:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753433938451 time 753476713723 state 0x0 ref 1 00000100:00100000:1.0:1789760769.230673:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753476974139 time 753476713723 state 0x1 ref 1 00000100:00100000:1.0:1789760769.230712:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753476974139 time 753551427854 state 0x0 ref 1 00000100:00100000:1.0:1789760769.230725:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495888512:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760769.230778:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25800704, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc809 00010000:00010000:1.0:1789760769.230788:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547409 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.230804:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760769.230810:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.230817:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547409 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.230832:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547409 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760769.282961:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25800704 -> 25804800 00010000:00010000:1.0:1789760769.282980:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547409 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.282995:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547409 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.283002:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547409 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.283009:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760769.283014:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547409 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.283023:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547409 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.283042:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547409 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.283053:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547409 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.283719:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495888512:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52988us (53807us total) trans 4294973616 rc 0/0 00000100:00100000:1.0:1789760769.283732:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6328 00000100:00100000:1.0:1789760769.283742:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753476974139 time 753551427854 state 0x2 ref 0 00000001:00080000:3.0:1789760769.284156:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973616 is committed 00000100:00100000:3.0:1789760769.291946:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760769.292320:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495888640 00000100:00100000:1.0:1789760769.292331:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495888640 00000100:00100000:1.0:1789760769.292385:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753476974139 time 753551427854 state 0x0 ref 1 00000100:00100000:1.0:1789760769.292400:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753551688270 time 753551427854 state 0x1 ref 1 00000100:00100000:1.0:1789760769.292432:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753551688270 time 753613149595 state 0x0 ref 1 00000100:00100000:1.0:1789760769.292445:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495888640:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760769.292497:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25804800, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc809 00010000:00010000:1.0:1789760769.292507:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547410 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.292523:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760769.292528:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.292534:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547410 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.292554:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547410 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.332625:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973617 is committed 00002000:00010000:2.0:1789760769.332755:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25804800 -> 25808896 00010000:00010000:2.0:1789760769.332772:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547410 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.332784:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547410 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.332794:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547410 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.332802:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760769.332806:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547410 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.332815:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547410 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.332844:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547410 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.332853:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547410 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.332951:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495888640:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40503us (41008us total) trans 4294973617 rc 0/0 00000100:00100000:2.0:1789760769.332961:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6329 00000100:00100000:2.0:1789760769.332971:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753551688270 time 753613149595 state 0x2 ref 0 00000100:00100000:2.0:1789760769.339242:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760769.339356:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495888768 00000100:00100000:0.0:1789760769.339365:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495888768 00000100:00100000:0.0:1789760769.339417:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753551688270 time 753613149595 state 0x0 ref 1 00000100:00100000:0.0:1789760769.339432:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753613410011 time 753613149595 state 0x1 ref 1 00000100:00100000:0.0:1789760769.339461:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753613410011 time 753660177784 state 0x0 ref 1 00000100:00100000:0.0:1789760769.339473:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495888768:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760769.339520:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25808896, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc809 00010000:00010000:0.0:1789760769.339530:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547417 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.339544:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760769.339549:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.339555:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547417 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.339567:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547417 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760769.412243:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973618 is committed 00002000:00010000:3.0:1789760769.412447:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25808896 -> 25812992 00010000:00010000:3.0:1789760769.412462:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547417 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.412475:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547417 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.412484:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547417 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.412492:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760769.412497:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547417 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.412506:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547417 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.412525:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547417 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.412532:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547417 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.412627:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495888768:12345-192.168.203.9@tcp:4:dd.0 Request processed in 73151us (73389us total) trans 4294973618 rc 0/0 00000100:00100000:3.0:1789760769.412637:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6330 00000100:00100000:3.0:1789760769.412647:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753613410011 time 753660177784 state 0x2 ref 0 00000100:00100000:3.0:1789760769.418651:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760769.418751:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495888896 00000100:00100000:1.0:1789760769.418761:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495888896 00000100:00100000:1.0:1789760769.418829:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753613410011 time 753660177784 state 0x0 ref 1 00000100:00100000:1.0:1789760769.418853:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753660438200 time 753660177784 state 0x1 ref 1 00000100:00100000:1.0:1789760769.418885:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753660438200 time 753739601444 state 0x0 ref 1 00000100:00100000:1.0:1789760769.418897:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495888896:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760769.418945:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25812992, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc809 00010000:00010000:1.0:1789760769.418954:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454741e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.418970:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760769.418974:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.418980:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454741e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.418993:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454741e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760769.516394:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973619 is committed 00002000:00010000:1.0:1789760769.517126:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25812992 -> 25817088 00010000:00010000:1.0:1789760769.517145:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454741e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.517159:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454741e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.517167:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454741e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.517175:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760769.517180:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454741e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.517189:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454741e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.517208:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454741e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.517216:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454741e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.517358:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495888896:12345-192.168.203.9@tcp:4:dd.0 Request processed in 98458us (98712us total) trans 4294973619 rc 0/0 00000100:00100000:1.0:1789760769.517369:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6331 00000100:00100000:1.0:1789760769.517380:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753660438200 time 753739601444 state 0x2 ref 0 00000100:00100000:3.0:1789760769.524027:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760769.524078:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760769.524123:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760769.524149:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760769.524168:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495889280 00000100:00100000:1.0:1789760769.524169:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495889408 00000100:00100000:1.0:1789760769.524177:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495889408 00000100:00100000:3.0:1789760769.524177:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495889280 00000100:00100000:3.0:1789760769.524223:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495889152 00000100:00100000:3.0:1789760769.524226:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495889152 00000100:00100000:1.0:1789760769.524233:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753660438200 time 753739601444 state 0x0 ref 1 00000100:00100000:3.0:1789760769.524246:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 510 00000100:00100000:1.0:1789760769.524247:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753739861860 time 753739601444 state 0x1 ref 1 00000100:00100000:3.0:1789760769.524255:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92901237ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2356:x1876699495889280:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:1.0:1789760769.524267:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753739861860 time 753844984401 state 0x0 ref 1 00000100:00100000:1.0:1789760769.524275:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92901237f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495889408:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760769.524349:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25817088, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc809 00000100:00100000:3.0:1789760769.524350:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92901237ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2356:x1876699495889280:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 96us (227us total) trans 0 rc 0/0 00010000:00010000:1.0:1789760769.524358:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547425 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.524358:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 510 00010000:00010000:1.0:1789760769.524374:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00000100:00100000:3.0:1789760769.524374:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 511 00010000:00010000:1.0:1789760769.524379:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00000100:00100000:3.0:1789760769.524380:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f921c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:17022965-1e4f-46fb-915c-61730096b8e2+8:2356:x1876699495889152:12345-192.168.203.9@tcp:400:kworker.0 00010000:00010000:1.0:1789760769.524386:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547425 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.524401:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547425 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.524403:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f921c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:17022965-1e4f-46fb-915c-61730096b8e2+8:2356:x1876699495889152:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 24us (254us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760769.524409:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 511 00000100:00080000:3.0:1789760769.524426:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760769.525006:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495889024 00000100:00100000:0.0:1789760769.525013:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495889024 00000100:00100000:0.0:1789760769.525057:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 412 00000100:00100000:0.0:1789760769.525064:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290122cec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2356:x1876699495889024:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:0.0:1789760769.525110:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290122cec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2356:x1876699495889024:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 45us (1030us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760769.525117:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 412 00000001:00080000:3.0:1789760769.576297:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973620 is committed 00002000:00010000:2.0:1789760769.576428:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25817088 -> 25821184 00010000:00010000:2.0:1789760769.576444:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547425 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.576456:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547425 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.576465:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547425 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.576472:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760769.576476:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547425 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.576484:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547425 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.576503:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547425 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.576510:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547425 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.576761:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92901237f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495889408:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52480us (52740us total) trans 4294973620 rc 0/0 00000100:00100000:2.0:1789760769.576772:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6332 00000100:00100000:2.0:1789760769.576783:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753739861860 time 753844984401 state 0x2 ref 0 00000100:00100000:2.0:1789760769.581755:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760769.581910:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495889536 00000100:00100000:3.0:1789760769.581920:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495889536 00000100:00100000:3.0:1789760769.581967:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753739861860 time 753844984401 state 0x0 ref 1 00000100:00100000:3.0:1789760769.581992:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753845244817 time 753844984401 state 0x1 ref 1 00000100:00100000:3.0:1789760769.582017:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753845244817 time 753902734693 state 0x0 ref 1 00000100:00100000:3.0:1789760769.582029:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495889536:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760769.582072:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25821184, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc809 00010000:00010000:3.0:1789760769.582080:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454742c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.582597:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:3.0:1789760769.582601:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.582608:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454742c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.582623:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454742c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760769.633263:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25821184 -> 25825280 00010000:00010000:3.0:1789760769.633304:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454742c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.633317:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454742c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.633325:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454742c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760769.634736:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973621 is committed 00010000:00010000:3.0:1789760769.634896:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760769.634903:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454742c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.634917:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454742c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.634938:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454742c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.634948:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454742c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.635102:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495889536:12345-192.168.203.9@tcp:4:dd.0 Request processed in 53069us (53351us total) trans 4294973621 rc 0/0 00000100:00100000:3.0:1789760769.635113:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6333 00000100:00100000:3.0:1789760769.635124:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753845244817 time 753902734693 state 0x2 ref 0 00000100:00100000:3.0:1789760769.640604:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760769.640693:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495889664 00000100:00100000:1.0:1789760769.640703:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495889664 00000100:00100000:1.0:1789760769.640749:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753845244817 time 753902734693 state 0x0 ref 1 00000100:00100000:1.0:1789760769.640761:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753902995109 time 753902734693 state 0x1 ref 1 00000100:00100000:1.0:1789760769.640826:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753902995109 time 753961542730 state 0x0 ref 1 00000100:00100000:1.0:1789760769.640849:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495889664:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760769.640897:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25825280, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc809 00010000:00010000:1.0:1789760769.640905:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547433 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.640918:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760769.640922:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.640927:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547433 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.640937:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547433 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760769.684319:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973622 is committed 00002000:00010000:1.0:1789760769.684529:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25825280 -> 25829376 00002000:00010000:1.0:1789760769.684534:0:13059:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760768 -> 1789760769 00002000:00010000:1.0:1789760769.684536:0:13059:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760768 -> 1789760769 00010000:00010000:1.0:1789760769.684549:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547433 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.684562:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547433 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.684570:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547433 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.684577:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760769.684582:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547433 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.684590:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547433 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.684612:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547433 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.684619:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547433 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.684712:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495889664:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43861us (44113us total) trans 4294973622 rc 0/0 00000100:00100000:1.0:1789760769.684722:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6334 00000100:00100000:1.0:1789760769.684732:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753902995109 time 753961542730 state 0x2 ref 0 00000100:00100000:1.0:1789760769.689020:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760769.689112:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495889792 00000100:00100000:2.0:1789760769.689122:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495889792 00000100:00100000:2.0:1789760769.689191:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753902995109 time 753961542730 state 0x0 ref 1 00000100:00100000:2.0:1789760769.689207:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753961803146 time 753961542730 state 0x1 ref 1 00000100:00100000:2.0:1789760769.689233:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753961803146 time 754009949725 state 0x0 ref 1 00000100:00100000:2.0:1789760769.689246:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495889792:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760769.689328:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25829376, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc809 00010000:00010000:2.0:1789760769.689350:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454743a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.689365:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:2.0:1789760769.689369:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.689377:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454743a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.689390:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454743a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760769.739644:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973623 is committed 00002000:00010000:3.0:1789760769.739757:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25829376 -> 25833472 00010000:00010000:3.0:1789760769.739774:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454743a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.739786:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454743a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.739795:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454743a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.739802:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760769.739806:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454743a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.739826:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454743a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.739860:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454743a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.739868:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x493674835454743a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.739960:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495889792:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50711us (50943us total) trans 4294973623 rc 0/0 00000100:00100000:3.0:1789760769.739971:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6335 00000100:00100000:3.0:1789760769.739981:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753961803146 time 754009949725 state 0x2 ref 0 00000100:00100000:2.0:1789760769.743678:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760769.743775:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495889920 00000100:00100000:3.0:1789760769.743786:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495889920 00000100:00100000:3.0:1789760769.743854:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 753961803146 time 754009949725 state 0x0 ref 1 00000100:00100000:3.0:1789760769.743869:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754010210141 time 754009949725 state 0x1 ref 1 00000100:00100000:3.0:1789760769.743931:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754010210141 time 754064647696 state 0x0 ref 1 00000100:00100000:3.0:1789760769.743944:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495889920:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760769.743991:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25833472, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc809 00010000:00010000:3.0:1789760769.743999:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547441 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.744014:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760769.744019:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.744026:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547441 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.744039:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547441 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760769.780635:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25833472 -> 25837568 00010000:00010000:3.0:1789760769.780655:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547441 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.780667:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547441 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.780677:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547441 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.780684:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760769.780689:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547441 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.780698:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547441 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.780717:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547441 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.780729:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547441 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.780871:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495889920:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36925us (37198us total) trans 4294973624 rc 0/0 00000100:00100000:3.0:1789760769.780881:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6336 00000100:00100000:3.0:1789760769.780891:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754010210141 time 754064647696 state 0x2 ref 0 00000001:00080000:0.0:1789760769.781031:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973624 is committed 00000100:00100000:0.0:1789760769.785613:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760769.785775:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495890048 00000100:00100000:1.0:1789760769.785784:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495890048 00000100:00100000:1.0:1789760769.785864:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754010210141 time 754064647696 state 0x0 ref 1 00000100:00100000:1.0:1789760769.785881:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754064908112 time 754064647696 state 0x1 ref 1 00000100:00100000:1.0:1789760769.785913:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754064908112 time 754106629683 state 0x0 ref 1 00000100:00100000:1.0:1789760769.785927:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495890048:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760769.785976:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25837568, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc809 00010000:00010000:1.0:1789760769.785985:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547448 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.786002:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760769.786007:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.786013:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547448 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.786028:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547448 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.820079:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973625 is committed 00002000:00010000:0.0:1789760769.820248:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25837568 -> 25841664 00010000:00010000:0.0:1789760769.820264:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547448 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.820276:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547448 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.820303:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547448 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.820311:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760769.820315:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547448 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.820324:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547448 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.820341:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547448 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.820349:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547448 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.820438:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495890048:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34508us (34830us total) trans 4294973625 rc 0/0 00000100:00100000:0.0:1789760769.820447:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6337 00000100:00100000:0.0:1789760769.820457:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754064908112 time 754106629683 state 0x2 ref 0 00000100:00100000:1.0:1789760769.826376:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760769.826652:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495890176 00000100:00100000:3.0:1789760769.826663:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495890176 00000100:00100000:3.0:1789760769.826715:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754064908112 time 754106629683 state 0x0 ref 1 00000100:00100000:3.0:1789760769.826731:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754106890099 time 754106629683 state 0x1 ref 1 00000100:00100000:3.0:1789760769.826801:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754106890099 time 754147516863 state 0x0 ref 1 00000100:00100000:3.0:1789760769.826815:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495890176:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760769.826881:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25841664, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc809 00010000:00010000:3.0:1789760769.826891:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454744f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.826907:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760769.826912:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.826918:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454744f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.826932:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454744f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760769.868884:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25841664 -> 25845760 00000001:00080000:0.0:1789760769.868898:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973626 is committed 00010000:00010000:3.0:1789760769.868904:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454744f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.869038:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454744f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.869050:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454744f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.869062:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760769.869067:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454744f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.869076:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454744f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.869096:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454744f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.869104:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454744f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.869186:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495890176:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42369us (42816us total) trans 4294973626 rc 0/0 00000100:00100000:3.0:1789760769.869197:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6338 00000100:00100000:3.0:1789760769.869206:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754106890099 time 754147516863 state 0x2 ref 0 00000100:00100000:2.0:1789760769.872862:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760769.872952:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495890304 00000100:00100000:3.0:1789760769.872963:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495890304 00000100:00100000:3.0:1789760769.873016:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754106890099 time 754147516863 state 0x0 ref 1 00000100:00100000:3.0:1789760769.873031:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754147777279 time 754147516863 state 0x1 ref 1 00000100:00100000:3.0:1789760769.873066:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754147777279 time 754193781904 state 0x0 ref 1 00000100:00100000:3.0:1789760769.873077:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034ce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495890304:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760769.873124:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25845760, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc809 00010000:00010000:3.0:1789760769.873133:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547456 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.873149:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760769.873152:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.873158:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547456 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.873171:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547456 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.886442:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760769.897255:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973627 is committed 00002000:00010000:0.0:1789760769.897337:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25845760 -> 25849856 00010000:00010000:0.0:1789760769.897353:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547456 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.897364:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547456 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.897372:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547456 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.897379:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760769.897383:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547456 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.897391:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547456 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.897415:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547456 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.897423:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547456 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.897590:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034ce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495890304:12345-192.168.203.9@tcp:4:dd.0 Request processed in 24509us (24730us total) trans 4294973627 rc 0/0 00000100:00100000:0.0:1789760769.897600:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6339 00000100:00100000:0.0:1789760769.897610:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754147777279 time 754193781904 state 0x2 ref 0 00000100:00100000:0.0:1789760769.901569:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760769.901787:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495890432 00000100:00100000:1.0:1789760769.902127:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495890432 00000100:00100000:1.0:1789760769.902521:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754147777279 time 754193781904 state 0x0 ref 1 00000100:00100000:1.0:1789760769.902999:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754194042320 time 754193781904 state 0x1 ref 1 00000100:00100000:1.0:1789760769.903033:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754194042320 time 754223749384 state 0x0 ref 1 00000100:00100000:1.0:1789760769.903048:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495890432:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760769.903101:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25849856, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc809 00010000:00010000:1.0:1789760769.903111:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x493674835454745d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.903128:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760769.903132:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.903139:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x493674835454745d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.903440:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x493674835454745d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760769.931351:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973628 is committed 00002000:00010000:0.0:1789760769.931364:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25849856 -> 25853952 00010000:00010000:0.0:1789760769.931379:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x493674835454745d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.931393:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x493674835454745d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.931402:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x493674835454745d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.931410:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760769.931415:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x493674835454745d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.931434:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x493674835454745d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.931456:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x493674835454745d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.931464:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x493674835454745d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.931605:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495890432:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28556us (30039us total) trans 4294973628 rc 0/0 00000100:00100000:1.0:1789760769.931616:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6340 00000100:00100000:1.0:1789760769.931625:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754194042320 time 754223749384 state 0x2 ref 0 00000100:00100000:2.0:1789760769.937905:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760769.937995:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495890560 00000100:00100000:3.0:1789760769.938004:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495890560 00000100:00100000:3.0:1789760769.938058:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754194042320 time 754223749384 state 0x0 ref 1 00000100:00100000:3.0:1789760769.938073:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754224009800 time 754223749384 state 0x1 ref 1 00000100:00100000:3.0:1789760769.938105:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754224009800 time 754258821558 state 0x0 ref 1 00000100:00100000:3.0:1789760769.938118:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495890560:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760769.938170:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25853952, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc809 00010000:00010000:3.0:1789760769.938180:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547464 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.938197:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760769.938202:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.938209:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547464 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.938223:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547464 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760769.971954:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25853952 -> 25858048 00010000:00010000:3.0:1789760769.971972:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547464 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.971991:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547464 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.972001:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547464 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.972007:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760769.972012:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547464 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.972021:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547464 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.972039:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547464 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.972047:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547464 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.972132:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495890560:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34013us (34231us total) trans 4294973629 rc 0/0 00000100:00100000:3.0:1789760769.972142:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6341 00000100:00100000:3.0:1789760769.972152:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754224009800 time 754258821558 state 0x2 ref 0 00000001:00080000:0.0:1789760769.972610:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973629 is committed 00000100:00100000:2.0:1789760769.987915:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760769.988012:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495890688 00000100:00100000:1.0:1789760769.988024:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495890688 00000100:00100000:1.0:1789760769.988085:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754224009800 time 754258821558 state 0x0 ref 1 00000100:00100000:1.0:1789760769.988101:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754259081974 time 754258821558 state 0x1 ref 1 00000100:00100000:1.0:1789760769.988130:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754259081974 time 754308846434 state 0x0 ref 1 00000100:00100000:1.0:1789760769.988142:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495890688:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760769.988197:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25858048, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc809 00010000:00010000:1.0:1789760769.988208:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454746b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.988225:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760769.988230:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.988237:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454746b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.988251:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454746b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.014577:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973630 is committed 00002000:00010000:0.0:1789760770.014931:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25858048 -> 25862144 00010000:00010000:0.0:1789760770.014946:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454746b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.014961:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454746b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.014971:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454746b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.014979:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760770.014983:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454746b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.014992:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454746b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.015012:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454746b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.015021:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454746b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.015324:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495890688:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27132us (27366us total) trans 4294973630 rc 0/0 00000100:00100000:0.0:1789760770.015336:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6342 00000100:00100000:0.0:1789760770.015347:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754259081974 time 754308846434 state 0x2 ref 0 00000100:00100000:2.0:1789760770.022657:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760770.022993:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495890816 00000100:00100000:1.0:1789760770.023002:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495890816 00000100:00100000:1.0:1789760770.023529:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754259081974 time 754308846434 state 0x0 ref 1 00000100:00100000:1.0:1789760770.023545:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754309106850 time 754308846434 state 0x1 ref 1 00000100:00100000:1.0:1789760770.023615:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754309106850 time 754344331752 state 0x0 ref 1 00000100:00100000:1.0:1789760770.023629:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04930380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495890816:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760770.023679:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25862144, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc809 00010000:00010000:1.0:1789760770.023688:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547472 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.023703:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760770.023708:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760770.023713:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547472 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.023728:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547472 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760770.076027:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973631 is committed 00002000:00010000:0.0:1789760770.076516:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25862144 -> 25866240 00010000:00010000:0.0:1789760770.076532:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547472 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.076547:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547472 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.076555:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547472 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.076563:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760770.076568:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547472 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.076581:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547472 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.076602:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547472 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.076610:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547472 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.076700:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04930380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495890816:12345-192.168.203.9@tcp:4:dd.0 Request processed in 53071us (54048us total) trans 4294973631 rc 0/0 00000100:00100000:0.0:1789760770.076711:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6343 00000100:00100000:0.0:1789760770.076721:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754309106850 time 754344331752 state 0x2 ref 0 00000100:00100000:2.0:1789760770.081799:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760770.081899:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495890944 00000100:00100000:3.0:1789760770.081909:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495890944 00000100:00100000:3.0:1789760770.081967:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754309106850 time 754344331752 state 0x0 ref 1 00000100:00100000:3.0:1789760770.081982:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754344592168 time 754344331752 state 0x1 ref 1 00000100:00100000:3.0:1789760770.082012:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754344592168 time 754402728924 state 0x0 ref 1 00000100:00100000:3.0:1789760770.082026:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04932d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495890944:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760770.082079:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25866240, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc809 00010000:00010000:3.0:1789760770.082089:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547479 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.082109:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760770.082114:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.082121:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547479 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.082137:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547479 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.129318:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973632 is committed 00002000:00010000:0.0:1789760770.131471:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25866240 -> 25870336 00010000:00010000:0.0:1789760770.131500:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547479 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.131517:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547479 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.131528:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547479 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.131537:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760770.131545:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547479 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.131556:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547479 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.131586:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547479 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.131595:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547479 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.131750:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04932d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495890944:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49718us (49954us total) trans 4294973632 rc 0/0 00000100:00100000:0.0:1789760770.132543:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6344 00000100:00100000:0.0:1789760770.132557:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754344592168 time 754402728924 state 0x2 ref 0 00000100:00100000:2.0:1789760770.137882:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760770.138122:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495891072 00000100:00100000:0.0:1789760770.138137:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495891072 00000100:00100000:0.0:1789760770.138222:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754344592168 time 754402728924 state 0x0 ref 1 00000100:00100000:0.0:1789760770.138242:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754402989340 time 754402728924 state 0x1 ref 1 00000100:00100000:0.0:1789760770.138311:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754402989340 time 754459026657 state 0x0 ref 1 00000100:00100000:0.0:1789760770.138325:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04933100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495891072:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760770.138390:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25870336, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc809 00010000:00010000:0.0:1789760770.138402:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547480 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.138422:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:0.0:1789760770.138428:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.138436:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547480 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.138452:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547480 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.186263:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973633 is committed 00002000:00010000:0.0:1789760770.186417:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25870336 -> 25874432 00010000:00010000:0.0:1789760770.186439:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547480 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.186453:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547480 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.186462:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547480 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.186670:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760770.186677:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547480 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.186689:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547480 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.186722:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547480 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.186732:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547480 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.187069:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04933100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495891072:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48738us (49197us total) trans 4294973633 rc 0/0 00000100:00100000:0.0:1789760770.187083:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6345 00000100:00100000:0.0:1789760770.187097:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754402989340 time 754459026657 state 0x2 ref 0 00000100:00100000:2.0:1789760770.191139:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760770.191424:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495891200 00000100:00100000:0.0:1789760770.191437:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495891200 00000100:00100000:0.0:1789760770.191581:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754402989340 time 754459026657 state 0x0 ref 1 00000100:00100000:0.0:1789760770.191603:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754459287073 time 754459026657 state 0x1 ref 1 00000100:00100000:0.0:1789760770.191677:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754459287073 time 754512392744 state 0x0 ref 1 00000100:00100000:0.0:1789760770.191693:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04933480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495891200:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760770.191779:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25874432, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc809 00010000:00010000:0.0:1789760770.191791:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547487 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.191822:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:0.0:1789760770.191837:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.191845:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547487 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.191862:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547487 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760770.257248:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973634 is committed 00002000:00010000:1.0:1789760770.257426:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25874432 -> 25878528 00010000:00010000:1.0:1789760770.257449:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547487 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.257464:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547487 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.257475:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547487 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.257483:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760770.257489:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547487 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.257500:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547487 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.257526:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547487 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.257534:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547487 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.257670:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04933480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495891200:12345-192.168.203.9@tcp:4:dd.0 Request processed in 65972us (66538us total) trans 4294973634 rc 0/0 00000100:00100000:1.0:1789760770.257682:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6346 00000100:00100000:1.0:1789760770.257695:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754459287073 time 754512392744 state 0x2 ref 0 00000100:00100000:2.0:1789760770.262027:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760770.262185:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495891328 00000100:00100000:0.0:1789760770.262200:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495891328 00000100:00100000:0.0:1789760770.262399:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754459287073 time 754512392744 state 0x0 ref 1 00000100:00100000:0.0:1789760770.262421:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754512653160 time 754512392744 state 0x1 ref 1 00000100:00100000:0.0:1789760770.265198:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754512653160 time 754585309640 state 0x0 ref 1 00000100:00100000:0.0:1789760770.265245:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04932680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495891328:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760770.265457:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25878528, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc809 00010000:00010000:0.0:1789760770.265469:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454748e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.265490:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:0.0:1789760770.265495:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.265503:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454748e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.265521:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454748e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760770.324489:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25878528 -> 25882624 00010000:00010000:0.0:1789760770.324518:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454748e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.324534:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454748e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.324545:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454748e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.324553:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760770.324559:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454748e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.324569:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454748e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.324595:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454748e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.324603:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454748e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.324738:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04932680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495891328:12345-192.168.203.9@tcp:4:dd.0 Request processed in 59507us (62722us total) trans 4294973635 rc 0/0 00000100:00100000:0.0:1789760770.324756:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6347 00000100:00100000:0.0:1789760770.324769:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754512653160 time 754585309640 state 0x2 ref 0 00000100:00100000:2.0:1789760770.328074:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760770.328222:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495891456 00000100:00100000:0.0:1789760770.328237:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495891456 00000100:00100000:0.0:1789760770.328349:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754512653160 time 754585309640 state 0x0 ref 1 00000100:00100000:0.0:1789760770.328372:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754585570056 time 754585309640 state 0x1 ref 1 00000100:00100000:0.0:1789760770.328405:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754585570056 time 754649121754 state 0x0 ref 1 00000100:00100000:0.0:1789760770.328420:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04930a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495891456:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760770.328486:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25882624, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc809 00010000:00010000:0.0:1789760770.328497:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547495 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.328517:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:0.0:1789760770.328522:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.328530:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547495 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.328547:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547495 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760770.337070:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973635 is committed 00002000:00010000:1.0:1789760770.360273:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25882624 -> 25886720 00010000:00010000:1.0:1789760770.360324:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547495 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.360341:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547495 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.360352:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547495 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.360361:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760770.360367:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547495 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.360379:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547495 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.360406:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547495 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.360421:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547495 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.360595:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04930a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495891456:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32171us (32531us total) trans 4294973636 rc 0/0 00000100:00100000:1.0:1789760770.360609:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6348 00000100:00100000:1.0:1789760770.360623:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754585570056 time 754649121754 state 0x2 ref 0 00000001:00080000:2.0:1789760770.360762:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973636 is committed 00000100:00100000:2.0:1789760770.364366:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760770.364472:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495891584 00000100:00100000:1.0:1789760770.364488:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495891584 00000100:00100000:1.0:1789760770.364566:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754585570056 time 754649121754 state 0x0 ref 1 00000100:00100000:1.0:1789760770.364585:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754649382170 time 754649121754 state 0x1 ref 1 00000100:00100000:1.0:1789760770.364620:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754649382170 time 754685336692 state 0x0 ref 1 00000100:00100000:1.0:1789760770.364637:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04931180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495891584:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760770.364708:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25886720, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc809 00010000:00010000:1.0:1789760770.364720:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454749c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.364741:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760770.364746:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760770.364755:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454749c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.364779:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454749c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760770.395721:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25886720 -> 25890816 00010000:00010000:2.0:1789760770.395753:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454749c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.395771:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454749c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.395783:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454749c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.395797:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760770.395813:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454749c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.395833:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454749c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.395859:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454749c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.395866:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454749c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.396008:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495891584:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31367us (31651us total) trans 4294973637 rc 0/0 00000100:00100000:2.0:1789760770.396022:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6349 00000100:00100000:2.0:1789760770.396035:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754649382170 time 754685336692 state 0x2 ref 0 00000001:00080000:2.0:1789760770.396098:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973637 is committed 00000100:00100000:0.0:1789760770.400204:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760770.401000:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495891712 00000100:00100000:3.0:1789760770.401017:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495891712 00000100:00100000:3.0:1789760770.401105:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754649382170 time 754685336692 state 0x0 ref 1 00000100:00100000:3.0:1789760770.401130:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754685597108 time 754685336692 state 0x1 ref 1 00000100:00100000:3.0:1789760770.401275:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754685597108 time 754721990149 state 0x0 ref 1 00000100:00100000:3.0:1789760770.401316:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495891712:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760770.401387:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25890816, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc809 00010000:00010000:3.0:1789760770.401399:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545474a3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.401420:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760770.401425:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.401433:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545474a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.401449:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545474a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760770.435648:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973638 is committed 00002000:00010000:0.0:1789760770.435799:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25890816 -> 25894912 00010000:00010000:0.0:1789760770.435840:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545474a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.435855:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545474a3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.435865:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545474a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.435873:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760770.435878:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545474a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.435888:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545474a3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.435912:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545474a3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.435921:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545474a3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.436054:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495891712:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34733us (35915us total) trans 4294973638 rc 0/0 00000100:00100000:0.0:1789760770.436065:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6350 00000100:00100000:0.0:1789760770.436078:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754685597108 time 754721990149 state 0x2 ref 0 00000100:00100000:0.0:1789760770.440503:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760770.440608:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495891840 00000100:00100000:1.0:1789760770.440621:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495891840 00000100:00100000:1.0:1789760770.440695:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754685597108 time 754721990149 state 0x0 ref 1 00000100:00100000:1.0:1789760770.440715:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754722250565 time 754721990149 state 0x1 ref 1 00000100:00100000:1.0:1789760770.440878:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754722250565 time 754761592651 state 0x0 ref 1 00000100:00100000:1.0:1789760770.440894:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495891840:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760770.440963:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25894912, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc809 00010000:00010000:1.0:1789760770.440974:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545474aa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.440995:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760770.441001:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760770.441011:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545474aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.441028:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545474aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760770.520718:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973639 is committed 00002000:00010000:0.0:1789760770.522507:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25894912 -> 25899008 00010000:00010000:0.0:1789760770.522537:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545474aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.522554:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545474aa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.522565:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545474aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.522574:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760770.522579:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545474aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.522591:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545474aa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.522616:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545474aa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.522625:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545474aa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.522770:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495891840:12345-192.168.203.9@tcp:4:dd.0 Request processed in 81872us (82275us total) trans 4294973639 rc 0/0 00000100:00100000:0.0:1789760770.522789:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6351 00000100:00100000:0.0:1789760770.522803:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754722250565 time 754761592651 state 0x2 ref 0 00000100:00100000:2.0:1789760770.526027:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760770.526190:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495891968 00000100:00100000:1.0:1789760770.526203:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495891968 00000100:00100000:1.0:1789760770.526322:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754722250565 time 754761592651 state 0x0 ref 1 00000100:00100000:1.0:1789760770.526342:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754761853067 time 754761592651 state 0x1 ref 1 00000100:00100000:1.0:1789760770.526379:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754761853067 time 754847095302 state 0x0 ref 1 00000100:00100000:1.0:1789760770.526396:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036399450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495891968:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760770.526497:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25899008, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc809 00010000:00010000:1.0:1789760770.526514:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545474b1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.526536:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760770.526542:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760770.526550:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545474b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.526573:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545474b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.577240:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973640 is committed 00002000:00010000:0.0:1789760770.577639:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25899008 -> 25903104 00010000:00010000:0.0:1789760770.577660:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545474b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.577677:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545474b1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.577688:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545474b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.577697:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760770.577703:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545474b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.577714:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545474b1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.577743:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545474b1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.577750:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545474b1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.578047:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036399450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495891968:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51647us (52027us total) trans 4294973640 rc 0/0 00000100:00100000:0.0:1789760770.578061:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6352 00000100:00100000:0.0:1789760770.578075:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754761853067 time 754847095302 state 0x2 ref 0 00000100:00100000:0.0:1789760770.582367:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760770.582484:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495892096 00000100:00100000:3.0:1789760770.582496:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495892096 00000100:00100000:3.0:1789760770.582566:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754761853067 time 754847095302 state 0x0 ref 1 00000100:00100000:3.0:1789760770.582586:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754847355718 time 754847095302 state 0x1 ref 1 00000100:00100000:3.0:1789760770.582622:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754847355718 time 754903338770 state 0x0 ref 1 00000100:00100000:3.0:1789760770.582636:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495892096:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760770.582703:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25903104, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc809 00010000:00010000:3.0:1789760770.582714:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545474b8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.582736:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760770.582743:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.582750:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545474b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.582778:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545474b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760770.643187:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25903104 -> 25907200 00010000:00010000:2.0:1789760770.643220:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545474b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.643244:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545474b8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.643255:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545474b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.643263:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760770.643270:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545474b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.643316:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545474b8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.643343:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545474b8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.643352:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545474b8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.643475:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495892096:12345-192.168.203.9@tcp:4:dd.0 Request processed in 60835us (61258us total) trans 4294973641 rc 0/0 00000100:00100000:2.0:1789760770.643488:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6353 00000100:00100000:2.0:1789760770.643499:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754847355718 time 754903338770 state 0x2 ref 0 00000001:00080000:2.0:1789760770.644465:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973641 is committed 00000100:00100000:2.0:1789760770.647443:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760770.647578:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495892224 00000100:00100000:0.0:1789760770.647591:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495892224 00000100:00100000:0.0:1789760770.647658:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754847355718 time 754903338770 state 0x0 ref 1 00000100:00100000:0.0:1789760770.647676:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754903599186 time 754903338770 state 0x1 ref 1 00000100:00100000:0.0:1789760770.647730:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754903599186 time 754968446806 state 0x0 ref 1 00000100:00100000:0.0:1789760770.647742:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04931f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495892224:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760770.647802:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25907200, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc809 00010000:00010000:0.0:1789760770.647813:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545474bf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.647841:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:0.0:1789760770.647847:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.647855:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545474bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.647877:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545474bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.720174:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973642 is committed 00002000:00010000:3.0:1789760770.721638:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25907200 -> 25911296 00002000:00010000:3.0:1789760770.721647:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760769 -> 1789760770 00002000:00010000:3.0:1789760770.721650:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760769 -> 1789760770 00010000:00010000:3.0:1789760770.721667:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545474bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.721683:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545474bf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.721693:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545474bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.721700:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760770.721706:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545474bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.721723:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545474bf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.721752:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545474bf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.721759:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545474bf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.722049:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495892224:12345-192.168.203.9@tcp:4:dd.0 Request processed in 74300us (74614us total) trans 4294973642 rc 0/0 00000100:00100000:3.0:1789760770.722063:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6354 00000100:00100000:3.0:1789760770.722077:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754903599186 time 754968446806 state 0x2 ref 0 00000100:00100000:1.0:1789760770.725510:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760770.725691:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495892352 00000100:00100000:0.0:1789760770.725706:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495892352 00000100:00100000:0.0:1789760770.725790:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754903599186 time 754968446806 state 0x0 ref 1 00000100:00100000:0.0:1789760770.725811:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754968707222 time 754968446806 state 0x1 ref 1 00000100:00100000:0.0:1789760770.725989:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754968707222 time 755046628852 state 0x0 ref 1 00000100:00100000:0.0:1789760770.726008:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495892352:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760770.726079:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25911296, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc809 00010000:00010000:0.0:1789760770.726092:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545474c6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.726113:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760770.726118:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.726127:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545474c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.726144:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545474c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760770.764324:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973643 is committed 00002000:00010000:1.0:1789760770.764452:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25911296 -> 25915392 00010000:00010000:1.0:1789760770.764473:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545474c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.764488:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545474c6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.764503:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545474c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.764511:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760770.764517:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545474c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.764528:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545474c6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.764555:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545474c6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.764563:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545474c6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.764728:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495892352:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38717us (39224us total) trans 4294973643 rc 0/0 00000100:00100000:1.0:1789760770.764740:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6355 00000100:00100000:1.0:1789760770.764753:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754968707222 time 755046628852 state 0x2 ref 0 00000100:00100000:3.0:1789760770.768865:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760770.770223:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495892480 00000100:00100000:1.0:1789760770.770243:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495892480 00000100:00100000:1.0:1789760770.770351:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 754968707222 time 755046628852 state 0x0 ref 1 00000100:00100000:1.0:1789760770.770371:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755046889268 time 755046628852 state 0x1 ref 1 00000100:00100000:1.0:1789760770.770398:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755046889268 time 755091115245 state 0x0 ref 1 00000100:00100000:1.0:1789760770.770415:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495892480:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760770.770482:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25915392, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc809 00010000:00010000:1.0:1789760770.770494:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545474cd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.770514:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:1.0:1789760770.770519:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760770.770526:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545474cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.770553:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545474cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.838088:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973644 is committed 00002000:00010000:2.0:1789760770.838334:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25915392 -> 25919488 00010000:00010000:2.0:1789760770.838357:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545474cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.838375:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545474cd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.838385:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545474cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.838394:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760770.838399:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545474cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.838410:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545474cd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.838436:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545474cd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.838445:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x49367483545474cd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.838579:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495892480:12345-192.168.203.9@tcp:4:dd.0 Request processed in 68160us (69723us total) trans 4294973644 rc 0/0 00000100:00100000:2.0:1789760770.838596:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6356 00000100:00100000:2.0:1789760770.838618:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755046889268 time 755091115245 state 0x2 ref 0 00000100:00100000:2.0:1789760770.844615:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760770.844816:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495892608 00000100:00100000:3.0:1789760770.844838:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495892608 00000100:00100000:3.0:1789760770.844914:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755046889268 time 755091115245 state 0x0 ref 1 00000100:00100000:3.0:1789760770.844933:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755091375661 time 755091115245 state 0x1 ref 1 00000100:00100000:3.0:1789760770.845005:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755091375661 time 755165720666 state 0x0 ref 1 00000100:00100000:3.0:1789760770.845020:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495892608:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760770.845092:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25919488, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc809 00010000:00010000:3.0:1789760770.845104:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545474d4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.845125:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760770.845131:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.845140:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545474d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.845158:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545474d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760770.888741:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25919488 -> 25923584 00010000:00010000:0.0:1789760770.888769:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545474d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.888790:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545474d4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.888809:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545474d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.888818:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760770.888832:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545474d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.888843:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545474d4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.888869:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545474d4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.888877:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545474d4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.888995:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495892608:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43970us (44396us total) trans 4294973645 rc 0/0 00000100:00100000:0.0:1789760770.889008:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6357 00000100:00100000:0.0:1789760770.889021:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755091375661 time 755165720666 state 0x2 ref 0 00000001:00080000:3.0:1789760770.889340:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973645 is committed 00000100:00100000:3.0:1789760770.892800:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760770.892965:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495892736 00000100:00100000:1.0:1789760770.892979:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495892736 00000100:00100000:1.0:1789760770.893049:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755091375661 time 755165720666 state 0x0 ref 1 00000100:00100000:1.0:1789760770.893068:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755165981082 time 755165720666 state 0x1 ref 1 00000100:00100000:2.0:1789760770.893237:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755165981082 time 755213950626 state 0x0 ref 1 00000100:00100000:2.0:1789760770.893266:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495892736:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760770.895455:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25923584, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc809 00010000:00010000:3.0:1789760770.895476:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545474db lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.895498:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760770.895502:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.895509:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545474db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.895525:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545474db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.910516:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760770.961234:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973646 is committed 00002000:00010000:3.0:1789760770.961479:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25923584 -> 25927680 00010000:00010000:3.0:1789760770.961500:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545474db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.961515:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545474db lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.961525:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545474db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.961534:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760770.961541:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545474db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.961550:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545474db lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.961575:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545474db lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.961583:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545474db lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.961712:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495892736:12345-192.168.203.9@tcp:4:dd.0 Request processed in 68455us (68919us total) trans 4294973646 rc 0/0 00000100:00100000:3.0:1789760770.961724:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6358 00000100:00100000:3.0:1789760770.961738:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755165981082 time 755213950626 state 0x2 ref 0 00000100:00100000:1.0:1789760770.966675:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760770.966816:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495892864 00000100:00100000:2.0:1789760770.966842:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495892864 00000100:00100000:2.0:1789760770.966920:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755165981082 time 755213950626 state 0x0 ref 1 00000100:00100000:2.0:1789760770.966939:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755214211042 time 755213950626 state 0x1 ref 1 00000100:00100000:2.0:1789760770.966981:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755214211042 time 755287696407 state 0x0 ref 1 00000100:00100000:2.0:1789760770.966996:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495892864:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760770.967063:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25927680, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc809 00010000:00010000:2.0:1789760770.967075:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545474e2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.967105:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760770.967111:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.967118:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545474e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.967139:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545474e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760771.028735:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25927680 -> 25931776 00010000:00010000:3.0:1789760771.028765:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545474e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.028778:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545474e2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.028788:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545474e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.028795:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760771.028938:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545474e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.028949:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545474e2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.028969:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545474e2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.028977:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545474e2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.029065:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495892864:12345-192.168.203.9@tcp:4:dd.0 Request processed in 62068us (62401us total) trans 4294973647 rc 0/0 00000100:00100000:3.0:1789760771.029077:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6359 00000100:00100000:3.0:1789760771.029087:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755214211042 time 755287696407 state 0x2 ref 0 00000001:00080000:0.0:1789760771.029457:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973647 is committed 00000100:00100000:2.0:1789760771.031380:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760771.031586:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495892992 00000100:00100000:0.0:1789760771.031596:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495892992 00000100:00100000:0.0:1789760771.031705:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755214211042 time 755287696407 state 0x0 ref 1 00000100:00100000:0.0:1789760771.031721:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755287956823 time 755287696407 state 0x1 ref 1 00000100:00100000:0.0:1789760771.031758:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755287956823 time 755352475220 state 0x0 ref 1 00000100:00100000:0.0:1789760771.031771:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04930380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495892992:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760771.031830:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25931776, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc809 00010000:00010000:0.0:1789760771.031839:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545474e9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.031855:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760771.031860:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.031867:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545474e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.031888:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545474e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760771.032449:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760771.070033:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973648 is committed 00002000:00010000:3.0:1789760771.070135:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25931776 -> 25935872 00010000:00010000:3.0:1789760771.070151:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545474e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.070165:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545474e9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.070173:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545474e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.070181:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760771.070186:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545474e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.070195:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545474e9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.070214:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545474e9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.070429:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545474e9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.070532:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04930380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495892992:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38759us (39154us total) trans 4294973648 rc 0/0 00000100:00100000:3.0:1789760771.070545:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6360 00000100:00100000:3.0:1789760771.070555:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755287956823 time 755352475220 state 0x2 ref 0 00000100:00100000:0.0:1789760771.075381:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760771.075565:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495893120 00000100:00100000:1.0:1789760771.075574:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495893120 00000100:00100000:1.0:1789760771.075624:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755287956823 time 755352475220 state 0x0 ref 1 00000100:00100000:1.0:1789760771.075639:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755352735636 time 755352475220 state 0x1 ref 1 00000100:00100000:1.0:1789760771.075658:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755352735636 time 755396375247 state 0x0 ref 1 00000100:00100000:1.0:1789760771.075670:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495893120:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760771.075715:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25935872, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc809 00010000:00010000:1.0:1789760771.075723:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545474f0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.075738:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:1.0:1789760771.075743:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.075749:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545474f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.075762:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545474f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.116655:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973649 is committed 00002000:00010000:0.0:1789760771.116758:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25935872 -> 25939968 00010000:00010000:0.0:1789760771.116772:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545474f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.116785:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545474f0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.116794:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545474f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.116811:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760771.116815:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545474f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.116834:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545474f0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.116853:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545474f0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.116860:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545474f0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.116946:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495893120:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41274us (41569us total) trans 4294973649 rc 0/0 00000100:00100000:0.0:1789760771.116957:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6361 00000100:00100000:0.0:1789760771.116967:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755352735636 time 755396375247 state 0x2 ref 0 00000100:00100000:0.0:1789760771.120582:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760771.120726:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495893248 00000100:00100000:3.0:1789760771.120736:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495893248 00000100:00100000:3.0:1789760771.120790:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755352735636 time 755396375247 state 0x0 ref 1 00000100:00100000:3.0:1789760771.120817:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755396635663 time 755396375247 state 0x1 ref 1 00000100:00100000:3.0:1789760771.120849:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755396635663 time 755441565733 state 0x0 ref 1 00000100:00100000:3.0:1789760771.120861:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495893248:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760771.120903:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25939968, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc809 00010000:00010000:3.0:1789760771.120912:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545474f7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.120927:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760771.120931:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.120938:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545474f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.120951:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545474f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.162804:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973650 is committed 00002000:00010000:3.0:1789760771.162880:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25939968 -> 25944064 00010000:00010000:3.0:1789760771.162897:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545474f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.162909:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545474f7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.162919:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545474f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.162985:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760771.162991:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545474f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.163002:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545474f7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.163021:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545474f7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.163029:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545474f7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.163122:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495893248:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42259us (42546us total) trans 4294973650 rc 0/0 00000100:00100000:3.0:1789760771.163134:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6362 00000100:00100000:3.0:1789760771.163144:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755396635663 time 755441565733 state 0x2 ref 0 00000100:00100000:3.0:1789760771.176781:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760771.177971:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495893376 00000100:00100000:0.0:1789760771.177983:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495893376 00000100:00100000:0.0:1789760771.178042:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755396635663 time 755441565733 state 0x0 ref 1 00000100:00100000:0.0:1789760771.178057:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755441826149 time 755441565733 state 0x1 ref 1 00000100:00100000:0.0:1789760771.178103:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755441826149 time 755498819448 state 0x0 ref 1 00000100:00100000:0.0:1789760771.178117:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495893376:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760771.178174:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25944064, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc809 00010000:00010000:0.0:1789760771.178183:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545474fe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.178198:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760771.178203:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.178209:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545474fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.178222:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545474fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760771.213953:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25944064 -> 25948160 00010000:00010000:0.0:1789760771.213974:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545474fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.213987:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545474fe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.213997:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545474fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.214005:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94aa00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760771.214010:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545474fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.214018:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545474fe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.214037:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545474fe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.214045:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x49367483545474fe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.214171:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495893376:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36051us (37394us total) trans 4294973651 rc 0/0 00000100:00100000:0.0:1789760771.214183:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6363 00000100:00100000:0.0:1789760771.214198:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755441826149 time 755498819448 state 0x2 ref 0 00000001:00080000:0.0:1789760771.214239:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973651 is committed 00000100:00100000:0.0:1789760771.217343:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760771.217518:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495893504 00000100:00100000:3.0:1789760771.217527:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495893504 00000100:00100000:3.0:1789760771.217580:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755441826149 time 755498819448 state 0x0 ref 1 00000100:00100000:3.0:1789760771.217600:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755499079864 time 755498819448 state 0x1 ref 1 00000100:00100000:3.0:1789760771.217633:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755499079864 time 755538349471 state 0x0 ref 1 00000100:00100000:3.0:1789760771.217646:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495893504:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760771.217693:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25948160, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc809 00010000:00010000:3.0:1789760771.217702:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547505 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.217717:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760771.217722:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.217728:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547505 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.217742:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547505 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760771.251963:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25948160 -> 25952256 00000001:00080000:1.0:1789760771.251974:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973652 is committed 00010000:00010000:0.0:1789760771.251982:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547505 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.251994:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547505 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.252004:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547505 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.252012:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760771.252016:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547505 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.252025:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547505 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.252044:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547505 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.252052:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547505 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.252137:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495893504:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34490us (34798us total) trans 4294973652 rc 0/0 00000100:00100000:0.0:1789760771.252155:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6364 00000100:00100000:0.0:1789760771.252164:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755499079864 time 755538349471 state 0x2 ref 0 00000100:00100000:1.0:1789760771.255840:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760771.255970:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495893632 00000100:00100000:3.0:1789760771.255980:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495893632 00000100:00100000:3.0:1789760771.256033:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755499079864 time 755538349471 state 0x0 ref 1 00000100:00100000:3.0:1789760771.256056:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755538609887 time 755538349471 state 0x1 ref 1 00000100:00100000:3.0:1789760771.256098:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755538609887 time 755576810760 state 0x0 ref 1 00000100:00100000:3.0:1789760771.256111:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495893632:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760771.256169:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25952256, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc809 00010000:00010000:3.0:1789760771.256183:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454750c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.256199:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760771.256203:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.256209:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454750c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.256223:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454750c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760771.285894:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973653 is committed 00002000:00010000:3.0:1789760771.286102:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25952256 -> 25956352 00010000:00010000:3.0:1789760771.286117:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454750c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.286129:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454750c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.286137:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454750c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.286144:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760771.286149:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454750c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.286158:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454750c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.286177:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454750c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.286184:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454750c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.286273:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495893632:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30160us (30438us total) trans 4294973653 rc 0/0 00000100:00100000:3.0:1789760771.286320:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6365 00000100:00100000:3.0:1789760771.286330:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755538609887 time 755576810760 state 0x2 ref 0 00000100:00100000:3.0:1789760771.290872:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760771.291335:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495893760 00000100:00100000:0.0:1789760771.291346:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495893760 00000100:00100000:0.0:1789760771.291400:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755538609887 time 755576810760 state 0x0 ref 1 00000100:00100000:0.0:1789760771.291416:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755577071176 time 755576810760 state 0x1 ref 1 00000100:00100000:0.0:1789760771.291449:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755577071176 time 755612165717 state 0x0 ref 1 00000100:00100000:0.0:1789760771.291461:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495893760:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760771.291507:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25956352, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc809 00010000:00010000:0.0:1789760771.291515:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547513 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.291531:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760771.291535:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.291546:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547513 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.291560:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547513 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760771.320028:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25956352 -> 25960448 00010000:00010000:0.0:1789760771.320050:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547513 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.320061:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547513 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.320071:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547513 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.320078:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ae00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760771.320083:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547513 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.320091:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547513 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.320109:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547513 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.320117:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547513 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.320231:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495893760:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28766us (29363us total) trans 4294973654 rc 0/0 00000100:00100000:0.0:1789760771.320241:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6366 00000100:00100000:0.0:1789760771.320252:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755577071176 time 755612165717 state 0x2 ref 0 00000001:00080000:1.0:1789760771.321023:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973654 is committed 00000100:00100000:0.0:1789760771.325258:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760771.325393:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495893888 00000100:00100000:3.0:1789760771.325402:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495893888 00000100:00100000:3.0:1789760771.325455:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755577071176 time 755612165717 state 0x0 ref 1 00000100:00100000:3.0:1789760771.325470:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755612426133 time 755612165717 state 0x1 ref 1 00000100:00100000:3.0:1789760771.325501:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755612426133 time 755646217936 state 0x0 ref 1 00000100:00100000:3.0:1789760771.325515:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495893888:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760771.325566:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25960448, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc809 00010000:00010000:3.0:1789760771.325575:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454751a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.325591:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760771.325596:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.325601:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454751a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.325617:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454751a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.358464:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789760771.358487:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff92903f926d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520313088:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760771.358609:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760771.358724:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520313088 00000100:00100000:2.0:1789760771.358733:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520313088 00000100:00100000:2.0:1789760771.358853:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 211 00000100:00100000:2.0:1789760771.358863:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520313088:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760771.358892:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527892480 pending 1703936 free 3832545280 avail 3817865216 00000100:00100000:2.0:1789760771.358939:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520313088:12345-0@lo:13:osp-pre-0-0.0 Request processed in 77us (330us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760771.358958:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 211 00000100:00080000:0.0:1789760771.366757:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760771.366958:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff92903f926d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520313088:0@lo:13:osp-pre-0-0.0 00000001:00080000:0.0:1789760771.383303:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973655 is committed 00002000:00010000:2.0:1789760771.383528:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25960448 -> 25964544 00010000:00010000:2.0:1789760771.383544:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454751a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.383556:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454751a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.383565:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454751a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.383572:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760771.383577:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454751a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.383586:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454751a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.383605:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454751a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.383614:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454751a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.383697:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495893888:12345-192.168.203.9@tcp:4:dd.0 Request processed in 58181us (58445us total) trans 4294973655 rc 0/0 00000100:00100000:2.0:1789760771.383707:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6367 00000100:00100000:2.0:1789760771.383717:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755612426133 time 755646217936 state 0x2 ref 0 00000100:00100000:2.0:1789760771.388934:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760771.389043:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495894016 00000100:00100000:3.0:1789760771.389053:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495894016 00000100:00100000:3.0:1789760771.389106:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755612426133 time 755646217936 state 0x0 ref 1 00000100:00100000:3.0:1789760771.389120:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755646478352 time 755646217936 state 0x1 ref 1 00000100:00100000:3.0:1789760771.389155:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755646478352 time 755709872188 state 0x0 ref 1 00000100:00100000:3.0:1789760771.389167:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04932d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495894016:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760771.389213:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25964544, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc809 00010000:00010000:3.0:1789760771.389222:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547521 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.389237:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760771.389242:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.389249:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547521 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.389263:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547521 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.446838:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973656 is committed 00002000:00010000:1.0:1789760771.447019:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25964544 -> 25968640 00010000:00010000:1.0:1789760771.447035:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547521 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.447049:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547521 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.447059:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547521 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.447066:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760771.447071:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547521 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.447080:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547521 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.447099:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547521 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.447107:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547521 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.447238:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04932d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495894016:12345-192.168.203.9@tcp:4:dd.0 Request processed in 58068us (58308us total) trans 4294973656 rc 0/0 00000100:00100000:1.0:1789760771.447249:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6368 00000100:00100000:1.0:1789760771.447258:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755646478352 time 755709872188 state 0x2 ref 0 00000100:00100000:0.0:1789760771.451564:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760771.451614:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495894144 00000100:00100000:3.0:1789760771.451625:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495894144 00000100:00100000:3.0:1789760771.451680:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755646478352 time 755709872188 state 0x0 ref 1 00000100:00100000:3.0:1789760771.451695:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755710132604 time 755709872188 state 0x1 ref 1 00000100:00100000:3.0:1789760771.451780:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755710132604 time 755772495564 state 0x0 ref 1 00000100:00100000:3.0:1789760771.451794:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495894144:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760771.451913:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25968640, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc809 00010000:00010000:3.0:1789760771.451922:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547528 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.451936:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760771.451941:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.451948:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547528 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.451962:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547528 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.488765:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973657 is committed 00002000:00010000:3.0:1789760771.488906:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25968640 -> 25972736 00010000:00010000:3.0:1789760771.488920:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547528 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.488930:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547528 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.488937:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547528 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.488943:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760771.488947:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547528 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.488955:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547528 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.488973:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547528 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.488980:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547528 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.489068:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495894144:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37272us (37507us total) trans 4294973657 rc 0/0 00000100:00100000:3.0:1789760771.489080:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6369 00000100:00100000:3.0:1789760771.489091:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755710132604 time 755772495564 state 0x2 ref 0 00000100:00100000:0.0:1789760771.494494:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760771.494730:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495894272 00000100:00100000:2.0:1789760771.494740:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495894272 00000100:00100000:2.0:1789760771.494796:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755710132604 time 755772495564 state 0x0 ref 1 00000100:00100000:2.0:1789760771.494829:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755772755980 time 755772495564 state 0x1 ref 1 00000100:00100000:2.0:1789760771.494906:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755772755980 time 755815621922 state 0x0 ref 1 00000100:00100000:2.0:1789760771.494923:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495894272:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760771.494976:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25972736, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc809 00010000:00010000:2.0:1789760771.494985:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454752f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.495001:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760771.495006:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.495012:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454752f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.495026:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454752f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760771.535048:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973658 is committed 00002000:00010000:3.0:1789760771.535210:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25972736 -> 25976832 00010000:00010000:3.0:1789760771.535225:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454752f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.535238:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454752f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.535247:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454752f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.535254:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760771.535259:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454752f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.535269:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454752f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.535351:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454752f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.535361:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454752f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.535487:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495894272:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40566us (40996us total) trans 4294973658 rc 0/0 00000100:00100000:3.0:1789760771.535498:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6370 00000100:00100000:3.0:1789760771.535509:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755772755980 time 755815621922 state 0x2 ref 0 00000100:00100000:2.0:1789760771.539021:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760771.539328:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495894400 00000100:00100000:1.0:1789760771.539338:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495894400 00000100:00100000:1.0:1789760771.539392:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755772755980 time 755815621922 state 0x0 ref 1 00000100:00100000:1.0:1789760771.539411:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755815882338 time 755815621922 state 0x1 ref 1 00000100:00100000:1.0:1789760771.539444:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755815882338 time 755860160740 state 0x0 ref 1 00000100:00100000:1.0:1789760771.539462:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04933100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495894400:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760771.539508:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25976832, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc809 00010000:00010000:1.0:1789760771.539517:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547536 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.539534:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:1.0:1789760771.539539:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.539545:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547536 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.539560:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547536 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760771.563913:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973659 is committed 00002000:00010000:2.0:1789760771.564060:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25976832 -> 25980928 00010000:00010000:2.0:1789760771.564077:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547536 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.564090:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547536 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.564101:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547536 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.564109:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760771.564114:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547536 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.564130:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547536 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.564151:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547536 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.564159:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547536 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.564250:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04933100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495894400:12345-192.168.203.9@tcp:4:dd.0 Request processed in 24791us (25233us total) trans 4294973659 rc 0/0 00000100:00100000:2.0:1789760771.564260:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6371 00000100:00100000:2.0:1789760771.564269:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755815882338 time 755860160740 state 0x2 ref 0 00000100:00100000:0.0:1789760771.569614:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760771.569958:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495894528 00000100:00100000:3.0:1789760771.569968:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495894528 00000100:00100000:3.0:1789760771.570025:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755815882338 time 755860160740 state 0x0 ref 1 00000100:00100000:3.0:1789760771.570040:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755860421156 time 755860160740 state 0x1 ref 1 00000100:00100000:3.0:1789760771.570072:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755860421156 time 755890788958 state 0x0 ref 1 00000100:00100000:3.0:1789760771.570085:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495894528:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760771.570136:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25980928, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc809 00010000:00010000:3.0:1789760771.570144:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454753d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.570160:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760771.570165:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.570171:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454753d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.570186:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454753d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760771.616788:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973660 is committed 00002000:00010000:2.0:1789760771.616848:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25980928 -> 25985024 00010000:00010000:2.0:1789760771.616864:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454753d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.616877:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454753d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.616887:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454753d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.616895:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760771.616900:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454753d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.616910:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454753d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.616930:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454753d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.616939:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454753d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.617038:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495894528:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46949us (47428us total) trans 4294973660 rc 0/0 00000100:00100000:2.0:1789760771.617049:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6372 00000100:00100000:2.0:1789760771.617060:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755860421156 time 755890788958 state 0x2 ref 0 00000100:00100000:1.0:1789760771.622201:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760771.622331:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495894656 00000100:00100000:3.0:1789760771.622341:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495894656 00000100:00100000:3.0:1789760771.622395:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755860421156 time 755890788958 state 0x0 ref 1 00000100:00100000:3.0:1789760771.622410:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755891049374 time 755890788958 state 0x1 ref 1 00000100:00100000:3.0:1789760771.622435:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755891049374 time 755943151413 state 0x0 ref 1 00000100:00100000:3.0:1789760771.622446:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03096680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495894656:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760771.622495:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25985024, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc809 00010000:00010000:3.0:1789760771.622504:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547544 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.622519:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760771.622524:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.622530:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547544 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.622543:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547544 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760771.655413:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973661 is committed 00002000:00010000:2.0:1789760771.655560:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25985024 -> 25989120 00002000:00010000:2.0:1789760771.655565:0:6714:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760770 -> 1789760771 00002000:00010000:2.0:1789760771.655568:0:6714:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760770 -> 1789760771 00010000:00010000:2.0:1789760771.655583:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547544 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.655595:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547544 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.655605:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547544 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.655614:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760771.655619:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547544 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.655629:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547544 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.655650:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547544 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.655659:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547544 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.655770:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03096680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495894656:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33319us (33571us total) trans 4294973661 rc 0/0 00000100:00100000:2.0:1789760771.655782:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6373 00000100:00100000:2.0:1789760771.656391:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755891049374 time 755943151413 state 0x2 ref 0 00000100:00100000:0.0:1789760771.662847:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760771.662955:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495894784 00000100:00100000:2.0:1789760771.662965:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495894784 00000100:00100000:2.0:1789760771.663020:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755891049374 time 755943151413 state 0x0 ref 1 00000100:00100000:2.0:1789760771.663035:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755943411829 time 755943151413 state 0x1 ref 1 00000100:00100000:2.0:1789760771.663067:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755943411829 time 755983783892 state 0x0 ref 1 00000100:00100000:2.0:1789760771.663084:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495894784:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760771.663129:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25989120, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc809 00010000:00010000:2.0:1789760771.663137:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454754b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.663152:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760771.663156:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.663162:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454754b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.663176:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454754b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760771.719876:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25989120 -> 25993216 00010000:00010000:2.0:1789760771.719896:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454754b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.719909:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454754b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.719918:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454754b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.719926:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760771.719931:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454754b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.719941:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454754b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.719974:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454754b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.719983:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454754b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760771.720130:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973662 is committed 00000100:00100000:2.0:1789760771.720273:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+6:2357:x1876699495894784:12345-192.168.203.9@tcp:4:dd.0 Request processed in 57186us (57430us total) trans 4294973662 rc 0/0 00000100:00100000:2.0:1789760771.720317:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6374 00000100:00100000:2.0:1789760771.720327:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755943411829 time 755983783892 state 0x2 ref 0 00000100:00100000:3.0:1789760771.723842:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760771.723900:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495894912 00000100:00100000:2.0:1789760771.723910:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495894912 00000100:00100000:2.0:1789760771.724066:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755943411829 time 755983783892 state 0x0 ref 1 00000100:00100000:2.0:1789760771.724084:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755984044308 time 755983783892 state 0x1 ref 1 00000100:00100000:2.0:1789760771.724120:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755984044308 time 756044836982 state 0x0 ref 1 00000100:00100000:2.0:1789760771.724132:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495894912:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760771.724185:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25993216, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc809 00010000:00010000:2.0:1789760771.724194:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547552 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.724210:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760771.724215:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.724220:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547552 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.724232:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547552 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.778732:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973663 is committed 00002000:00010000:1.0:1789760771.780214:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25993216 -> 25997312 00010000:00010000:1.0:1789760771.780232:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547552 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.780249:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547552 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.780258:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547552 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.780265:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760771.780270:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547552 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.780297:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547552 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.780315:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547552 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.780321:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547552 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.780422:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495894912:12345-192.168.203.9@tcp:4:dd.0 Request processed in 56288us (56585us total) trans 4294973663 rc 0/0 00000100:00100000:1.0:1789760771.780431:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6375 00000100:00100000:1.0:1789760771.780441:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755984044308 time 756044836982 state 0x2 ref 0 00000100:00100000:0.0:1789760771.786596:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760771.786709:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495895040 00000100:00100000:3.0:1789760771.786719:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495895040 00000100:00100000:3.0:1789760771.786782:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 755984044308 time 756044836982 state 0x0 ref 1 00000100:00100000:3.0:1789760771.786916:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756045097398 time 756044836982 state 0x1 ref 1 00000100:00100000:3.0:1789760771.786945:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756045097398 time 756107662219 state 0x0 ref 1 00000100:00100000:3.0:1789760771.786957:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495895040:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760771.787006:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25997312, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc809 00010000:00010000:3.0:1789760771.787015:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547559 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.787030:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760771.787034:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.787041:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547559 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.787055:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547559 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.826867:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973664 is committed 00002000:00010000:1.0:1789760771.827118:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25997312 -> 26001408 00010000:00010000:1.0:1789760771.827135:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547559 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.827148:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547559 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.827158:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547559 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.827165:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760771.827170:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547559 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.827179:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547559 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.827199:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547559 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.827207:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547559 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.827503:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495895040:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40542us (40914us total) trans 4294973664 rc 0/0 00000100:00100000:1.0:1789760771.827514:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6376 00000100:00100000:1.0:1789760771.827525:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756045097398 time 756107662219 state 0x2 ref 0 00000100:00100000:0.0:1789760771.836726:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760771.836859:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495895168 00000100:00100000:2.0:1789760771.836868:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495895168 00000100:00100000:2.0:1789760771.836999:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756045097398 time 756107662219 state 0x0 ref 1 00000100:00100000:2.0:1789760771.837020:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756107922635 time 756107662219 state 0x1 ref 1 00000100:00100000:2.0:1789760771.837091:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756107922635 time 756157807082 state 0x0 ref 1 00000100:00100000:2.0:1789760771.837104:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495895168:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760771.837152:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26001408, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc809 00010000:00010000:2.0:1789760771.837161:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547560 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.837177:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760771.837181:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.837188:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547560 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.837202:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547560 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.892041:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973665 is committed 00002000:00010000:2.0:1789760771.892214:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26001408 -> 26005504 00010000:00010000:2.0:1789760771.892229:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547560 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.892242:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547560 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.892250:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547560 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.892257:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760771.892261:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547560 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.892270:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547560 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.892304:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547560 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.892312:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547560 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.892399:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495895168:12345-192.168.203.9@tcp:4:dd.0 Request processed in 55294us (55679us total) trans 4294973665 rc 0/0 00000100:00100000:2.0:1789760771.892410:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6377 00000100:00100000:2.0:1789760771.892421:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756107922635 time 756157807082 state 0x2 ref 0 00000100:00100000:0.0:1789760771.899421:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760771.899559:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495895296 00000100:00100000:1.0:1789760771.899570:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495895296 00000100:00100000:1.0:1789760771.899624:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756107922635 time 756157807082 state 0x0 ref 1 00000100:00100000:1.0:1789760771.899640:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756158067498 time 756157807082 state 0x1 ref 1 00000100:00100000:1.0:1789760771.899668:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756158067498 time 756220385335 state 0x0 ref 1 00000100:00100000:1.0:1789760771.899680:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495895296:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760771.899727:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26005504, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc809 00010000:00010000:1.0:1789760771.899735:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547567 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.899751:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:1.0:1789760771.899755:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.899763:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547567 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.899779:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547567 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.925444:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973666 is committed 00002000:00010000:2.0:1789760771.925599:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26005504 -> 26009600 00010000:00010000:2.0:1789760771.925614:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547567 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.925627:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547567 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.925636:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547567 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.925642:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760771.925648:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547567 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.925657:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547567 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.925676:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547567 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.925688:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547567 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.925768:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495895296:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26086us (26355us total) trans 4294973666 rc 0/0 00000100:00100000:2.0:1789760771.925777:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6378 00000100:00100000:2.0:1789760771.925786:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756158067498 time 756220385335 state 0x2 ref 0 00000100:00100000:2.0:1789760771.929510:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760771.929721:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495895424 00000100:00100000:1.0:1789760771.929731:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495895424 00000100:00100000:1.0:1789760771.929799:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756158067498 time 756220385335 state 0x0 ref 1 00000100:00100000:1.0:1789760771.929824:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756220645751 time 756220385335 state 0x1 ref 1 00000100:00100000:1.0:1789760771.929855:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756220645751 time 756250572004 state 0x0 ref 1 00000100:00100000:1.0:1789760771.929868:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04933480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495895424:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760771.929921:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26009600, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc809 00010000:00010000:1.0:1789760771.929939:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454756e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.929955:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:1.0:1789760771.929959:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.929965:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454756e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.929978:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454756e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.935170:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760771.959664:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26009600 -> 26013696 00010000:00010000:2.0:1789760771.959683:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454756e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.959697:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454756e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.959707:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454756e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.959714:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760771.959719:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454756e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.959728:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454756e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.959752:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454756e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.959760:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454756e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.959887:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04933480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495895424:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30016us (30385us total) trans 4294973667 rc 0/0 00000100:00100000:2.0:1789760771.959897:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6379 00000100:00100000:2.0:1789760771.959907:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756220645751 time 756250572004 state 0x2 ref 0 00000001:00080000:0.0:1789760771.960498:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973667 is committed 00000100:00100000:0.0:1789760771.970409:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760771.971027:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495895552 00000100:00100000:2.0:1789760771.971037:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495895552 00000100:00100000:2.0:1789760771.971088:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756220645751 time 756250572004 state 0x0 ref 1 00000100:00100000:2.0:1789760771.971103:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756250832420 time 756250572004 state 0x1 ref 1 00000100:00100000:2.0:1789760771.971135:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756250832420 time 756291852145 state 0x0 ref 1 00000100:00100000:2.0:1789760771.971147:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495895552:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760771.971194:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26013696, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc809 00010000:00010000:2.0:1789760771.971204:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547575 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.971221:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760771.971226:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.971232:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547575 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.971246:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547575 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.012210:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973668 is committed 00002000:00010000:2.0:1789760772.012455:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26013696 -> 26017792 00010000:00010000:2.0:1789760772.012471:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547575 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.012484:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547575 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.012493:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547575 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.012501:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760772.012506:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547575 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.012515:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547575 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.012535:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547575 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.012543:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547575 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.012638:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495895552:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41488us (42234us total) trans 4294973668 rc 0/0 00000100:00100000:2.0:1789760772.012648:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6380 00000100:00100000:2.0:1789760772.012658:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756250832420 time 756291852145 state 0x2 ref 0 00000100:00100000:1.0:1789760772.015904:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760772.016952:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495895680 00000100:00100000:2.0:1789760772.016963:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495895680 00000100:00100000:2.0:1789760772.017022:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756250832420 time 756291852145 state 0x0 ref 1 00000100:00100000:2.0:1789760772.017037:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756292112561 time 756291852145 state 0x1 ref 1 00000100:00100000:2.0:1789760772.017059:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756292112561 time 756337775702 state 0x0 ref 1 00000100:00100000:2.0:1789760772.017071:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495895680:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760772.017118:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26017792, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc809 00010000:00010000:2.0:1789760772.017127:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454757c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.017141:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760772.017145:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.017151:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454757c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.017164:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454757c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.048265:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973669 is committed 00002000:00010000:2.0:1789760772.050541:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26017792 -> 26021888 00010000:00010000:2.0:1789760772.050560:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454757c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.050574:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454757c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.050584:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454757c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.050592:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760772.050597:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454757c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.050608:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454757c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.050628:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454757c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.050636:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454757c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.050726:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495895680:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33652us (34828us total) trans 4294973669 rc 0/0 00000100:00100000:2.0:1789760772.050740:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6381 00000100:00100000:2.0:1789760772.050750:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756292112561 time 756337775702 state 0x2 ref 0 00000100:00100000:0.0:1789760772.058131:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760772.058420:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495895808 00000100:00100000:2.0:1789760772.058434:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495895808 00000100:00100000:3.0:1789760772.058608:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756292112561 time 756337775702 state 0x0 ref 1 00000100:00100000:3.0:1789760772.058624:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756338036118 time 756337775702 state 0x1 ref 1 00000100:00100000:3.0:1789760772.058680:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756338036118 time 756379397252 state 0x0 ref 1 00000100:00100000:3.0:1789760772.058691:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495895808:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760772.058764:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26021888, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc809 00010000:00010000:3.0:1789760772.058775:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547583 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.058804:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760772.058810:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.058824:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547583 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.058841:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547583 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.098841:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973670 is committed 00002000:00010000:0.0:1789760772.099088:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26021888 -> 26025984 00010000:00010000:0.0:1789760772.099113:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547583 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.099130:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547583 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.099272:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547583 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.100013:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760772.100020:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547583 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.100035:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547583 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.100063:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547583 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.100077:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547583 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.101047:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495895808:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42343us (42916us total) trans 4294973670 rc 0/0 00000100:00100000:1.0:1789760772.101064:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6382 00000100:00100000:1.0:1789760772.101079:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756338036118 time 756379397252 state 0x2 ref 0 00000100:00100000:0.0:1789760772.104553:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760772.104983:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495895936 00000100:00100000:2.0:1789760772.105066:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495895936 00000100:00100000:2.0:1789760772.105145:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756338036118 time 756379397252 state 0x0 ref 1 00000100:00100000:2.0:1789760772.105398:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756379657668 time 756379397252 state 0x1 ref 1 00000100:00100000:2.0:1789760772.105481:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756379657668 time 756426196423 state 0x0 ref 1 00000100:00100000:2.0:1789760772.105497:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495895936:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760772.105566:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26025984, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc809 00010000:00010000:2.0:1789760772.105577:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454758a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.105597:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760772.105603:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.105610:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454758a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.105627:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454758a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.136465:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973671 is committed 00002000:00010000:2.0:1789760772.136712:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26025984 -> 26030080 00010000:00010000:2.0:1789760772.136734:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454758a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.136748:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454758a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.136760:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454758a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.136769:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760772.136774:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454758a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.136783:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454758a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.136883:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454758a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.136893:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454758a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.137054:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495895936:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31554us (32507us total) trans 4294973671 rc 0/0 00000100:00100000:2.0:1789760772.137070:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6383 00000100:00100000:2.0:1789760772.137083:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756379657668 time 756426196423 state 0x2 ref 0 00000100:00100000:2.0:1789760772.142048:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760772.142160:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495896064 00000100:00100000:1.0:1789760772.142174:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495896064 00000100:00100000:1.0:1789760772.142263:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756379657668 time 756426196423 state 0x0 ref 1 00000100:00100000:1.0:1789760772.142407:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756426456839 time 756426196423 state 0x1 ref 1 00000100:00100000:1.0:1789760772.142439:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756426456839 time 756463155135 state 0x0 ref 1 00000100:00100000:1.0:1789760772.142453:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04932680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495896064:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760772.142520:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26030080, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc809 00010000:00010000:1.0:1789760772.142531:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547591 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.142552:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:1.0:1789760772.142558:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.142566:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547591 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.142583:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547591 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.173070:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973672 is committed 00002000:00010000:2.0:1789760772.173272:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26030080 -> 26034176 00010000:00010000:2.0:1789760772.173315:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547591 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.173332:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547591 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.173342:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547591 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.173351:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760772.173357:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547591 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.173368:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547591 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.173392:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547591 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.173400:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547591 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.173530:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04932680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495896064:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31073us (31491us total) trans 4294973672 rc 0/0 00000100:00100000:2.0:1789760772.173542:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6384 00000100:00100000:2.0:1789760772.173555:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756426456839 time 756463155135 state 0x2 ref 0 00000100:00100000:2.0:1789760772.176519:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760772.176625:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495896192 00000100:00100000:1.0:1789760772.176639:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495896192 00000100:00100000:1.0:1789760772.176717:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756426456839 time 756463155135 state 0x0 ref 1 00000100:00100000:1.0:1789760772.176736:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756463415551 time 756463155135 state 0x1 ref 1 00000100:00100000:1.0:1789760772.176774:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756463415551 time 756497490256 state 0x0 ref 1 00000100:00100000:1.0:1789760772.176798:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04930a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495896192:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760772.176875:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26034176, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc809 00010000:00010000:1.0:1789760772.176886:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547598 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.176907:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:1.0:1789760772.176912:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.176920:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547598 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.176937:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547598 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.206675:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973673 is committed 00002000:00010000:1.0:1789760772.206887:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26034176 -> 26038272 00010000:00010000:1.0:1789760772.206907:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547598 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.206923:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547598 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.206935:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547598 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.206943:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760772.206949:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547598 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.206960:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547598 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.206984:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547598 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.206993:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547598 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.207249:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04930a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495896192:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30455us (30736us total) trans 4294973673 rc 0/0 00000100:00100000:1.0:1789760772.207264:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6385 00000100:00100000:1.0:1789760772.207327:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756463415551 time 756497490256 state 0x2 ref 0 00000100:00100000:3.0:1789760772.211132:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760772.211311:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495896320 00000100:00100000:2.0:1789760772.211325:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495896320 00000100:00100000:2.0:1789760772.211409:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756463415551 time 756497490256 state 0x0 ref 1 00000100:00100000:2.0:1789760772.211428:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756497750672 time 756497490256 state 0x1 ref 1 00000100:00100000:2.0:1789760772.211463:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756497750672 time 756532179533 state 0x0 ref 1 00000100:00100000:2.0:1789760772.211476:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495896320:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760772.211541:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26038272, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc809 00010000:00010000:2.0:1789760772.211551:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454759f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.211569:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760772.211574:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.211581:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454759f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.211597:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454759f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.263324:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973674 is committed 00002000:00010000:3.0:1789760772.263584:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26038272 -> 26042368 00010000:00010000:3.0:1789760772.263605:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454759f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.263622:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454759f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.263633:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454759f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.263641:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760772.263647:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454759f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.263657:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454759f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.263682:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454759f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.263691:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x493674835454759f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.263916:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495896320:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52434us (52793us total) trans 4294973674 rc 0/0 00000100:00100000:3.0:1789760772.263930:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6386 00000100:00100000:3.0:1789760772.263942:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756497750672 time 756532179533 state 0x2 ref 0 00000100:00100000:0.0:1789760772.266917:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760772.267019:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495896448 00000100:00100000:2.0:1789760772.267033:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495896448 00000100:00100000:2.0:1789760772.267109:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756497750672 time 756532179533 state 0x0 ref 1 00000100:00100000:2.0:1789760772.267128:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756532439949 time 756532179533 state 0x1 ref 1 00000100:00100000:2.0:1789760772.267235:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756532439949 time 756587950542 state 0x0 ref 1 00000100:00100000:2.0:1789760772.267251:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495896448:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760772.267564:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26042368, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc809 00010000:00010000:0.0:1789760772.267577:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545475a6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.267597:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:0.0:1789760772.267603:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.267611:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545475a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.267629:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545475a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760772.294527:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26042368 -> 26046464 00010000:00010000:0.0:1789760772.294556:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545475a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.294572:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545475a6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.294589:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545475a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.294597:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760772.294603:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545475a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.294612:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545475a6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.294639:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545475a6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.294648:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x49367483545475a6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.294776:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495896448:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27521us (27865us total) trans 4294973675 rc 0/0 00000100:00100000:0.0:1789760772.296782:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6387 00000100:00100000:0.0:1789760772.296806:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756532439949 time 756587950542 state 0x2 ref 0 00000001:00080000:0.0:1789760772.296989:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973675 is committed 00000100:00100000:2.0:1789760772.297967:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760772.298460:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495896576 00000100:00100000:1.0:1789760772.298475:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495896576 00000100:00100000:1.0:1789760772.298549:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756532439949 time 756587950542 state 0x0 ref 1 00000100:00100000:1.0:1789760772.298567:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756588210958 time 756587950542 state 0x1 ref 1 00000100:00100000:1.0:1789760772.298607:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756588210958 time 756619322157 state 0x0 ref 1 00000100:00100000:1.0:1789760772.298621:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04931180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495896576:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760772.298685:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26046464, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc809 00010000:00010000:1.0:1789760772.298696:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475ad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.298717:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:1.0:1789760772.298723:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.298732:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.298748:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760772.374659:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26046464 -> 26050560 00010000:00010000:1.0:1789760772.374680:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.374693:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475ad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.374702:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.374709:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760772.374716:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.374724:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475ad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.374742:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475ad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.374750:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475ad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.374855:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495896576:12345-192.168.203.9@tcp:4:dd.0 Request processed in 76232us (76894us total) trans 4294973676 rc 0/0 00000100:00100000:1.0:1789760772.374866:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6388 00000100:00100000:1.0:1789760772.374875:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756588210958 time 756619322157 state 0x2 ref 0 00000001:00080000:2.0:1789760772.375093:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973676 is committed 00000100:00100000:2.0:1789760772.383515:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760772.383594:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495896704 00000100:00100000:3.0:1789760772.383604:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495896704 00000100:00100000:3.0:1789760772.383661:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756588210958 time 756619322157 state 0x0 ref 1 00000100:00100000:3.0:1789760772.383679:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756619582573 time 756619322157 state 0x1 ref 1 00000100:00100000:3.0:1789760772.383713:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756619582573 time 756704429167 state 0x0 ref 1 00000100:00100000:3.0:1789760772.383726:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495896704:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760772.383773:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26050560, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc809 00010000:00010000:3.0:1789760772.383781:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545475b4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.383795:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760772.383799:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.383803:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545475b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.383827:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545475b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.426736:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973677 is committed 00002000:00010000:0.0:1789760772.426908:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26050560 -> 26054656 00010000:00010000:0.0:1789760772.426925:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545475b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.426943:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545475b4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.426952:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545475b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.426960:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760772.426965:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545475b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.426975:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545475b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.426998:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545475b4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.427005:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545475b4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.427107:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495896704:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43378us (43595us total) trans 4294973677 rc 0/0 00000100:00100000:0.0:1789760772.427117:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6389 00000100:00100000:0.0:1789760772.427128:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756619582573 time 756704429167 state 0x2 ref 0 00000100:00100000:0.0:1789760772.437438:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760772.437607:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495896832 00000100:00100000:1.0:1789760772.437617:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495896832 00000100:00100000:1.0:1789760772.437678:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756619582573 time 756704429167 state 0x0 ref 1 00000100:00100000:1.0:1789760772.437693:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756704689583 time 756704429167 state 0x1 ref 1 00000100:00100000:1.0:1789760772.437724:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756704689583 time 756758441080 state 0x0 ref 1 00000100:00100000:1.0:1789760772.437736:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495896832:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760772.437786:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26054656, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc809 00010000:00010000:1.0:1789760772.437805:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475bb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.437829:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:1.0:1789760772.437834:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.437840:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.437854:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.476271:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973678 is committed 00002000:00010000:1.0:1789760772.476433:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26054656 -> 26058752 00010000:00010000:1.0:1789760772.476448:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.476461:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475bb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.476470:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.476478:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760772.476483:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.476492:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475bb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.476512:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475bb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.476519:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475bb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.476612:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495896832:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38872us (39179us total) trans 4294973678 rc 0/0 00000100:00100000:1.0:1789760772.476625:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6390 00000100:00100000:1.0:1789760772.476635:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756704689583 time 756758441080 state 0x2 ref 0 00000100:00100000:2.0:1789760772.483638:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760772.483745:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495896960 00000100:00100000:1.0:1789760772.483755:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495896960 00000100:00100000:1.0:1789760772.483823:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756704689583 time 756758441080 state 0x0 ref 1 00000100:00100000:1.0:1789760772.483840:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756758701496 time 756758441080 state 0x1 ref 1 00000100:00100000:1.0:1789760772.483869:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756758701496 time 756804586412 state 0x0 ref 1 00000100:00100000:1.0:1789760772.483881:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495896960:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760772.483930:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26058752, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc809 00010000:00010000:1.0:1789760772.483938:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475c2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.483955:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:1.0:1789760772.483959:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.483966:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.483985:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.519822:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973679 is committed 00002000:00010000:2.0:1789760772.520012:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26058752 -> 26062848 00010000:00010000:2.0:1789760772.520029:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.520043:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475c2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.520053:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.520061:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760772.520065:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.520074:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475c2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.520094:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475c2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.520104:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545475c2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.520242:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495896960:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36359us (36610us total) trans 4294973679 rc 0/0 00000100:00100000:2.0:1789760772.520252:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6391 00000100:00100000:2.0:1789760772.520263:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756758701496 time 756804586412 state 0x2 ref 0 00000100:00100000:3.0:1789760772.522683:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760772.526423:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495897088 00000100:00100000:0.0:1789760772.526437:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495897088 00000100:00100000:0.0:1789760772.526500:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756758701496 time 756804586412 state 0x0 ref 1 00000100:00100000:0.0:1789760772.526516:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756804846828 time 756804586412 state 0x1 ref 1 00000100:00100000:0.0:1789760772.526548:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756804846828 time 756847264799 state 0x0 ref 1 00000100:00100000:0.0:1789760772.526561:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495897088:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760772.526609:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26062848, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc809 00010000:00010000:0.0:1789760772.526618:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475c9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.526634:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:0.0:1789760772.526639:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.526645:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.526659:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.560016:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973680 is committed 00002000:00010000:0.0:1789760772.560116:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26062848 -> 26066944 00010000:00010000:0.0:1789760772.560131:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.560145:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475c9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.560155:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.560161:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760772.560167:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.560176:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475c9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.560194:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475c9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.560202:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475c9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.560396:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495897088:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33832us (37718us total) trans 4294973680 rc 0/0 00000100:00100000:0.0:1789760772.560406:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6392 00000100:00100000:0.0:1789760772.560416:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756804846828 time 756847264799 state 0x2 ref 0 00000100:00100000:3.0:1789760772.565090:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760772.565481:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495897216 00000100:00100000:1.0:1789760772.565555:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495897216 00000100:00100000:1.0:1789760772.565611:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756804846828 time 756847264799 state 0x0 ref 1 00000100:00100000:1.0:1789760772.565626:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756847525215 time 756847264799 state 0x1 ref 1 00000100:00100000:1.0:1789760772.565657:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756847525215 time 756886374253 state 0x0 ref 1 00000100:00100000:1.0:1789760772.565670:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495897216:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760772.565717:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26066944, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc809 00010000:00010000:1.0:1789760772.565726:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545475d0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.565742:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:1.0:1789760772.565746:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.565752:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545475d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.565766:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545475d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.591528:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973681 is committed 00002000:00010000:2.0:1789760772.591629:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26066944 -> 26071040 00010000:00010000:2.0:1789760772.591644:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545475d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.591656:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545475d0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.591665:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545475d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.591674:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760772.591679:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545475d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.591688:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545475d0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.591707:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545475d0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.591714:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545475d0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.591816:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495897216:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26137us (26724us total) trans 4294973681 rc 0/0 00000100:00100000:2.0:1789760772.591827:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6393 00000100:00100000:2.0:1789760772.591837:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756847525215 time 756886374253 state 0x2 ref 0 00000100:00100000:0.0:1789760772.602223:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760772.602309:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495897344 00000100:00100000:2.0:1789760772.602321:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495897344 00000100:00100000:2.0:1789760772.602371:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756847525215 time 756886374253 state 0x0 ref 1 00000100:00100000:2.0:1789760772.602384:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756886634669 time 756886374253 state 0x1 ref 1 00000100:00100000:2.0:1789760772.602412:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756886634669 time 756923130216 state 0x0 ref 1 00000100:00100000:2.0:1789760772.602421:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495897344:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760772.602466:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26071040, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc809 00010000:00010000:2.0:1789760772.602474:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545475d7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.602487:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760772.602501:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.602508:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545475d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.602521:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545475d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.629157:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973682 is committed 00002000:00010000:2.0:1789760772.629268:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26071040 -> 26075136 00010000:00010000:2.0:1789760772.629312:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545475d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.629325:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545475d7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.629333:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545475d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.629340:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760772.629345:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545475d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.629353:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545475d7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.629377:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545475d7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.629384:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545475d7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.629478:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495897344:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27051us (27260us total) trans 4294973682 rc 0/0 00000100:00100000:2.0:1789760772.629488:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6394 00000100:00100000:2.0:1789760772.629498:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756886634669 time 756923130216 state 0x2 ref 0 00000100:00100000:3.0:1789760772.634183:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760772.634339:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495897472 00000100:00100000:0.0:1789760772.634350:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495897472 00000100:00100000:0.0:1789760772.634408:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756886634669 time 756923130216 state 0x0 ref 1 00000100:00100000:0.0:1789760772.634423:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756923390632 time 756923130216 state 0x1 ref 1 00000100:00100000:0.0:1789760772.634451:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756923390632 time 756955168235 state 0x0 ref 1 00000100:00100000:0.0:1789760772.634464:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495897472:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760772.634509:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26075136, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc809 00010000:00010000:0.0:1789760772.634518:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475de lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.634533:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:0.0:1789760772.634537:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.634544:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.634558:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.665315:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973683 is committed 00002000:00010000:0.0:1789760772.665422:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26075136 -> 26079232 00002000:00010000:0.0:1789760772.665427:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760771 -> 1789760772 00002000:00010000:0.0:1789760772.665430:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760771 -> 1789760772 00010000:00010000:0.0:1789760772.665444:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.665456:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475de lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.665469:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.665475:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760772.665480:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.665490:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.665509:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475de lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.665516:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475de lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.665611:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495897472:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31145us (31435us total) trans 4294973683 rc 0/0 00000100:00100000:0.0:1789760772.665622:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6395 00000100:00100000:0.0:1789760772.665634:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756923390632 time 756955168235 state 0x2 ref 0 00000100:00100000:3.0:1789760772.678567:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760772.678677:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495897600 00000100:00100000:2.0:1789760772.678687:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495897600 00000100:00100000:2.0:1789760772.678742:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756923390632 time 756955168235 state 0x0 ref 1 00000100:00100000:2.0:1789760772.678770:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756955428651 time 756955168235 state 0x1 ref 1 00000100:00100000:2.0:1789760772.678806:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756955428651 time 756999523015 state 0x0 ref 1 00000100:00100000:2.0:1789760772.678828:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495897600:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760772.678876:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26079232, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc809 00010000:00010000:2.0:1789760772.678886:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545475e5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.678902:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760772.678907:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.678914:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545475e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.678927:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545475e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.714711:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973684 is committed 00002000:00010000:2.0:1789760772.714805:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26079232 -> 26083328 00010000:00010000:2.0:1789760772.714832:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545475e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.714845:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545475e5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.714854:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545475e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.714861:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760772.714865:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545475e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.714874:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545475e5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.714897:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545475e5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.714910:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545475e5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.717402:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495897600:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38564us (38834us total) trans 4294973684 rc 0/0 00000100:00100000:2.0:1789760772.717418:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6396 00000100:00100000:2.0:1789760772.717429:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756955428651 time 756999523015 state 0x2 ref 0 00000100:00100000:3.0:1789760772.723230:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760772.723448:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495897728 00000100:00100000:2.0:1789760772.723458:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495897728 00000100:00100000:2.0:1789760772.723516:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756955428651 time 756999523015 state 0x0 ref 1 00000100:00100000:2.0:1789760772.723532:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756999783431 time 756999523015 state 0x1 ref 1 00000100:00100000:2.0:1789760772.723612:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756999783431 time 757044328008 state 0x0 ref 1 00000100:00100000:2.0:1789760772.723626:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495897728:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760772.723675:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26083328, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc809 00010000:00010000:2.0:1789760772.723685:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545475ec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.723701:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760772.723705:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.723716:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545475ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.723736:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545475ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.753086:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973685 is committed 00002000:00010000:2.0:1789760772.753229:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26083328 -> 26087424 00010000:00010000:2.0:1789760772.753244:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545475ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.753258:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545475ec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.753269:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545475ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.753293:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760772.753299:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545475ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.753310:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545475ec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.753329:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545475ec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.753337:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545475ec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.753474:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495897728:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29846us (30257us total) trans 4294973685 rc 0/0 00000100:00100000:2.0:1789760772.753482:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6397 00000100:00100000:2.0:1789760772.753490:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756999783431 time 757044328008 state 0x2 ref 0 00000100:00100000:3.0:1789760772.759023:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760772.759155:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495897856 00000100:00100000:0.0:1789760772.759165:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495897856 00000100:00100000:0.0:1789760772.759223:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 756999783431 time 757044328008 state 0x0 ref 1 00000100:00100000:0.0:1789760772.759238:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757044588424 time 757044328008 state 0x1 ref 1 00000100:00100000:0.0:1789760772.759273:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757044588424 time 757079989676 state 0x0 ref 1 00000100:00100000:0.0:1789760772.759508:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929018a80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495897856:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760772.759565:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26087424, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc809 00010000:00010000:0.0:1789760772.759575:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475f3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.759591:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760772.759595:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.759602:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.759615:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.783167:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973686 is committed 00002000:00010000:1.0:1789760772.783327:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26087424 -> 26091520 00010000:00010000:1.0:1789760772.783346:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.783365:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475f3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.783380:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.783388:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760772.783393:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.783402:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475f3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.783422:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475f3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.783429:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x49367483545475f3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.783524:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929018a80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495897856:12345-192.168.203.9@tcp:4:dd.0 Request processed in 24013us (24508us total) trans 4294973686 rc 0/0 00000100:00100000:1.0:1789760772.783534:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6398 00000100:00100000:1.0:1789760772.783544:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757044588424 time 757079989676 state 0x2 ref 0 00000100:00100000:3.0:1789760772.789862:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760772.791543:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495897984 00000100:00100000:2.0:1789760772.791555:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495897984 00000100:00100000:3.0:1789760772.792036:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757044588424 time 757079989676 state 0x0 ref 1 00000100:00100000:3.0:1789760772.792052:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757080250092 time 757079989676 state 0x1 ref 1 00000100:00100000:3.0:1789760772.792156:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757080250092 time 757112871160 state 0x0 ref 1 00000100:00100000:3.0:1789760772.792173:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036e57100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495897984:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760772.792226:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26091520, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc809 00010000:00010000:3.0:1789760772.792236:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545475fa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.792567:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760772.792571:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.792579:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545475fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.792593:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545475fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760772.828541:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26091520 -> 26095616 00010000:00010000:1.0:1789760772.828565:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545475fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.828585:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545475fa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.828600:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545475fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.828682:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760772.828687:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545475fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.828696:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545475fa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.828722:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545475fa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.828731:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545475fa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.828855:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036e57100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495897984:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36680us (38997us total) trans 4294973687 rc 0/0 00000100:00100000:1.0:1789760772.828867:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6399 00000001:00080000:0.0:1789760772.828975:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973687 is committed 00000100:00100000:1.0:1789760772.829709:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757080250092 time 757112871160 state 0x2 ref 0 00000100:00100000:3.0:1789760772.835967:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760772.836273:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495898112 00000100:00100000:2.0:1789760772.836318:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495898112 00000100:00100000:2.0:1789760772.836380:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757080250092 time 757112871160 state 0x0 ref 1 00000100:00100000:2.0:1789760772.836399:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757113131576 time 757112871160 state 0x1 ref 1 00000100:00100000:2.0:1789760772.836584:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757113131576 time 757157299684 state 0x0 ref 1 00000100:00100000:2.0:1789760772.836602:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036e55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495898112:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760772.836657:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26095616, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc809 00010000:00010000:2.0:1789760772.836666:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547601 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.836682:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760772.836686:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.836693:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547601 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.836707:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547601 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.875724:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973688 is committed 00002000:00010000:1.0:1789760772.876526:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26095616 -> 26099712 00010000:00010000:1.0:1789760772.876543:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547601 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.876556:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547601 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.876565:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547601 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.876573:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760772.876578:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547601 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.876587:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547601 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.876606:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547601 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.876614:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547601 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.876703:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036e55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495898112:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40099us (40799us total) trans 4294973688 rc 0/0 00000100:00100000:1.0:1789760772.876713:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6400 00000100:00100000:1.0:1789760772.876722:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757113131576 time 757157299684 state 0x2 ref 0 00000100:00100000:3.0:1789760772.881797:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760772.881919:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495898240 00000100:00100000:1.0:1789760772.881930:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495898240 00000100:00100000:1.0:1789760772.881983:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757113131576 time 757157299684 state 0x0 ref 1 00000100:00100000:1.0:1789760772.881998:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757157560100 time 757157299684 state 0x1 ref 1 00000100:00100000:1.0:1789760772.882511:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757157560100 time 757203225528 state 0x0 ref 1 00000100:00100000:1.0:1789760772.882526:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495898240:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760772.882715:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26099712, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc809 00010000:00010000:1.0:1789760772.882725:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547608 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.882742:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760772.882823:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.882831:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547608 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.882845:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547608 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.923375:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973689 is committed 00002000:00010000:1.0:1789760772.923604:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26099712 -> 26103808 00010000:00010000:1.0:1789760772.923621:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547608 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.923633:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547608 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.923642:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547608 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.923649:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760772.923654:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547608 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.923663:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547608 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.923683:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547608 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.923692:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547608 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.923839:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495898240:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41312us (42045us total) trans 4294973689 rc 0/0 00000100:00100000:1.0:1789760772.923851:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6401 00000100:00100000:1.0:1789760772.923861:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757157560100 time 757203225528 state 0x2 ref 0 00000100:00100000:3.0:1789760772.930243:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760772.930697:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495898368 00000100:00100000:0.0:1789760772.930708:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495898368 00000100:00100000:0.0:1789760772.930766:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757157560100 time 757203225528 state 0x0 ref 1 00000100:00100000:0.0:1789760772.930782:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757203485944 time 757203225528 state 0x1 ref 1 00000100:00100000:0.0:1789760772.930946:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757203485944 time 757251661228 state 0x0 ref 1 00000100:00100000:0.0:1789760772.930960:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495898368:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760772.931010:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26103808, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc809 00010000:00010000:0.0:1789760772.931019:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454760f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.931035:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760772.931040:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.931046:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454760f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.931060:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454760f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.959156:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760772.959253:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973690 is committed 00002000:00010000:3.0:1789760772.959391:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26103808 -> 26107904 00010000:00010000:3.0:1789760772.959406:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454760f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.959419:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454760f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.959427:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454760f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.959440:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760772.959445:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454760f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.959454:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454760f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.959477:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454760f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.959486:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454760f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.959581:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495898368:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28619us (29343us total) trans 4294973690 rc 0/0 00000100:00100000:3.0:1789760772.959591:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6402 00000100:00100000:3.0:1789760772.959601:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757203485944 time 757251661228 state 0x2 ref 0 00000100:00100000:0.0:1789760772.964212:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760772.964377:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495898496 00000100:00100000:1.0:1789760772.964386:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495898496 00000100:00100000:1.0:1789760772.964442:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757203485944 time 757251661228 state 0x0 ref 1 00000100:00100000:1.0:1789760772.964454:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757251921644 time 757251661228 state 0x1 ref 1 00000100:00100000:1.0:1789760772.964476:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757251921644 time 757285193969 state 0x0 ref 1 00000100:00100000:1.0:1789760772.964489:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495898496:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760772.964559:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26107904, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc809 00010000:00010000:1.0:1789760772.964568:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547616 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.964584:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760772.964588:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.964976:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547616 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.964994:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547616 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.999602:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973691 is committed 00002000:00010000:3.0:1789760772.999764:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26107904 -> 26112000 00010000:00010000:3.0:1789760772.999792:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547616 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.999815:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547616 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.999830:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547616 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.999837:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760772.999843:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547616 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.999853:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547616 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.999872:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547616 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.999880:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547616 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.999977:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495898496:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35482us (35767us total) trans 4294973691 rc 0/0 00000100:00100000:3.0:1789760772.999987:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6403 00000100:00100000:3.0:1789760772.999998:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757251921644 time 757285193969 state 0x2 ref 0 00000100:00100000:2.0:1789760773.003637:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760773.003882:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495898624 00000100:00100000:1.0:1789760773.003891:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495898624 00000100:00100000:1.0:1789760773.003961:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757251921644 time 757285193969 state 0x0 ref 1 00000100:00100000:1.0:1789760773.003977:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757285454385 time 757285193969 state 0x1 ref 1 00000100:00080000:0.0:1789760773.004054:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760773.004105:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757285454385 time 757324820480 state 0x0 ref 1 00000100:00100000:1.0:1789760773.004122:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495898624:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760773.004182:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26112000, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc809 00010000:00010000:1.0:1789760773.004190:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454761d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.004206:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760773.004210:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.004216:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454761d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.004229:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454761d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760773.056107:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26112000 -> 26116096 00010000:00010000:2.0:1789760773.056128:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454761d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.056141:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454761d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.056149:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454761d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.056156:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760773.056160:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454761d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.056171:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454761d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.056191:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454761d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.056198:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454761d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.056335:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495898624:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52210us (52702us total) trans 4294973692 rc 0/0 00000100:00100000:2.0:1789760773.056346:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6404 00000100:00100000:2.0:1789760773.056356:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757285454385 time 757324820480 state 0x2 ref 0 00000001:00080000:3.0:1789760773.056446:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973692 is committed 00000100:00100000:2.0:1789760773.059649:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760773.059728:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495898752 00000100:00100000:3.0:1789760773.059738:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495898752 00000100:00100000:3.0:1789760773.059793:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757285454385 time 757324820480 state 0x0 ref 1 00000100:00100000:3.0:1789760773.059970:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757325080896 time 757324820480 state 0x1 ref 1 00000100:00100000:3.0:1789760773.060047:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757325080896 time 757380763464 state 0x0 ref 1 00000100:00100000:3.0:1789760773.060061:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495898752:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760773.060108:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26116096, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc809 00010000:00010000:3.0:1789760773.060117:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547624 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.060132:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760773.060137:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.060143:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547624 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.060156:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547624 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.096205:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973693 is committed 00002000:00010000:3.0:1789760773.096327:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26116096 -> 26120192 00010000:00010000:3.0:1789760773.096342:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547624 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.096356:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547624 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.096365:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547624 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.096377:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760773.096382:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547624 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.096392:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547624 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.096412:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547624 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.096420:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547624 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.096514:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495898752:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36450us (36867us total) trans 4294973693 rc 0/0 00000100:00100000:3.0:1789760773.096524:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6405 00000100:00100000:3.0:1789760773.096535:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757325080896 time 757380763464 state 0x2 ref 0 00000100:00100000:0.0:1789760773.101609:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760773.101728:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495898880 00000100:00100000:3.0:1789760773.101739:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495898880 00000100:00100000:3.0:1789760773.101854:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757325080896 time 757380763464 state 0x0 ref 1 00000100:00100000:3.0:1789760773.101871:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757381023880 time 757380763464 state 0x1 ref 1 00000100:00100000:3.0:1789760773.101946:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757381023880 time 757422662006 state 0x0 ref 1 00000100:00100000:3.0:1789760773.101963:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495898880:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760773.102012:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26120192, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc809 00010000:00010000:3.0:1789760773.102021:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454762b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.102038:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760773.102043:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.102050:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454762b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.102065:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454762b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.130566:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973694 is committed 00002000:00010000:0.0:1789760773.130735:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26120192 -> 26124288 00010000:00010000:0.0:1789760773.130752:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454762b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.130765:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454762b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.130775:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454762b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.130853:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760773.130859:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454762b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.130868:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454762b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.130888:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454762b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.130896:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454762b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.130990:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495898880:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29025us (29386us total) trans 4294973694 rc 0/0 00000100:00100000:0.0:1789760773.131000:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6406 00000100:00100000:0.0:1789760773.131011:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757381023880 time 757422662006 state 0x2 ref 0 00000100:00100000:0.0:1789760773.136011:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760773.136114:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495899008 00000100:00100000:2.0:1789760773.136123:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495899008 00000100:00100000:2.0:1789760773.136177:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757381023880 time 757422662006 state 0x0 ref 1 00000100:00100000:2.0:1789760773.136193:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757422922422 time 757422662006 state 0x1 ref 1 00000100:00100000:2.0:1789760773.136225:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757422922422 time 757456940765 state 0x0 ref 1 00000100:00100000:2.0:1789760773.136238:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f872680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495899008:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760773.136324:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26124288, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc809 00010000:00010000:2.0:1789760773.136334:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547632 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.136353:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760773.136357:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.136364:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547632 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.136378:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547632 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.185046:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973695 is committed 00002000:00010000:0.0:1789760773.185583:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26124288 -> 26128384 00010000:00010000:0.0:1789760773.185599:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547632 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.185612:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547632 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.185621:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547632 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.185633:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760773.185638:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547632 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.185646:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547632 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.185665:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547632 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.185672:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547632 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.185762:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f872680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495899008:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49522us (49758us total) trans 4294973695 rc 0/0 00000100:00100000:0.0:1789760773.185772:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6407 00000100:00100000:0.0:1789760773.185787:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757422922422 time 757456940765 state 0x2 ref 0 00000100:00100000:2.0:1789760773.189206:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760773.189629:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495899136 00000100:00100000:3.0:1789760773.189640:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495899136 00000100:00100000:3.0:1789760773.189692:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757422922422 time 757456940765 state 0x0 ref 1 00000100:00100000:3.0:1789760773.189708:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757457201181 time 757456940765 state 0x1 ref 1 00000100:00100000:3.0:1789760773.189738:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757457201181 time 757510455143 state 0x0 ref 1 00000100:00100000:3.0:1789760773.189751:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495899136:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760773.189820:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26128384, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc809 00010000:00010000:3.0:1789760773.189830:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547639 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.189847:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760773.189851:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.189857:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547639 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.189881:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547639 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.238561:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973696 is committed 00002000:00010000:3.0:1789760773.238675:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26128384 -> 26132480 00010000:00010000:3.0:1789760773.238692:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547639 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.238705:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547639 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.238714:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547639 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.238722:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760773.238727:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547639 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.238736:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547639 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.238758:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547639 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.238765:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547639 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.238879:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495899136:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49124us (49677us total) trans 4294973696 rc 0/0 00000100:00100000:3.0:1789760773.238889:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6408 00000100:00100000:3.0:1789760773.238899:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757457201181 time 757510455143 state 0x2 ref 0 00000100:00100000:3.0:1789760773.246362:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760773.246471:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495899264 00000100:00100000:0.0:1789760773.246481:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495899264 00000100:00100000:0.0:1789760773.246536:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757457201181 time 757510455143 state 0x0 ref 1 00000100:00100000:0.0:1789760773.246552:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757510715559 time 757510455143 state 0x1 ref 1 00000100:00100000:0.0:1789760773.246589:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757510715559 time 757567305531 state 0x0 ref 1 00000100:00100000:0.0:1789760773.246602:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903639a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495899264:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760773.246650:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26132480, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc809 00010000:00010000:0.0:1789760773.246660:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547640 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.246678:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760773.246682:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.246690:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547640 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.246703:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547640 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760773.273158:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26132480 -> 26136576 00010000:00010000:0.0:1789760773.273178:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547640 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.273192:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547640 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.273200:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547640 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.273206:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760773.273210:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547640 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.273219:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547640 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.273238:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547640 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.273246:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547640 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.273369:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903639a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495899264:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26763us (27010us total) trans 4294973697 rc 0/0 00000100:00100000:0.0:1789760773.273379:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6409 00000100:00100000:0.0:1789760773.273389:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757510715559 time 757567305531 state 0x2 ref 0 00000001:00080000:3.0:1789760773.274874:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973697 is committed 00000100:00100000:0.0:1789760773.283402:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760773.283521:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495899392 00000100:00100000:2.0:1789760773.283533:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495899392 00000100:00100000:2.0:1789760773.283604:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757510715559 time 757567305531 state 0x0 ref 1 00000100:00100000:2.0:1789760773.283621:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757567565947 time 757567305531 state 0x1 ref 1 00000100:00100000:2.0:1789760773.283644:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757567565947 time 757604361324 state 0x0 ref 1 00000100:00100000:2.0:1789760773.283655:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f872680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495899392:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760773.283703:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26136576, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc809 00010000:00010000:2.0:1789760773.283711:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547647 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.283728:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760773.283732:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.283738:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547647 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.283751:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547647 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760773.315375:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973698 is committed 00002000:00010000:2.0:1789760773.315631:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26136576 -> 26140672 00010000:00010000:2.0:1789760773.315647:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547647 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.315661:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547647 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.315676:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547647 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.315683:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760773.315688:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547647 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.315697:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547647 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.315715:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547647 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.315724:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547647 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.315832:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f872680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495899392:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32174us (32435us total) trans 4294973698 rc 0/0 00000100:00100000:2.0:1789760773.315842:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6410 00000100:00100000:2.0:1789760773.315853:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757567565947 time 757604361324 state 0x2 ref 0 00000100:00100000:0.0:1789760773.326135:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760773.326375:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495899520 00000100:00100000:3.0:1789760773.326385:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495899520 00000100:00100000:3.0:1789760773.326442:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757567565947 time 757604361324 state 0x0 ref 1 00000100:00100000:3.0:1789760773.326456:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757604621740 time 757604361324 state 0x1 ref 1 00000100:00100000:3.0:1789760773.326564:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757604621740 time 757647278973 state 0x0 ref 1 00000100:00100000:3.0:1789760773.326579:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495899520:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760773.326626:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26140672, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc809 00010000:00010000:3.0:1789760773.326636:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454764e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.326657:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760773.326661:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.326667:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454764e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.326679:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454764e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760773.363595:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973699 is committed 00002000:00010000:0.0:1789760773.363705:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26140672 -> 26144768 00010000:00010000:0.0:1789760773.363720:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454764e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.363734:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454764e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.363744:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454764e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.363758:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760773.363763:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454764e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.363773:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454764e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.363959:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454764e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.363969:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454764e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.364095:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495899520:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37514us (37966us total) trans 4294973699 rc 0/0 00000100:00100000:0.0:1789760773.364106:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6411 00000100:00100000:0.0:1789760773.364117:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757604621740 time 757647278973 state 0x2 ref 0 00000100:00100000:0.0:1789760773.368933:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760773.369114:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495899648 00000100:00100000:3.0:1789760773.369124:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495899648 00000100:00100000:3.0:1789760773.369220:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757604621740 time 757647278973 state 0x0 ref 1 00000100:00100000:3.0:1789760773.369235:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757647539389 time 757647278973 state 0x1 ref 1 00000100:00100000:3.0:1789760773.369269:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757647539389 time 757689985703 state 0x0 ref 1 00000100:00100000:3.0:1789760773.369312:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495899648:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760773.369363:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26144768, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc809 00010000:00010000:3.0:1789760773.369374:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547655 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.369403:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760773.369408:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.369415:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547655 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.369435:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547655 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.403971:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973700 is committed 00002000:00010000:3.0:1789760773.404132:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26144768 -> 26148864 00010000:00010000:3.0:1789760773.404149:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547655 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.404161:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547655 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.404172:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547655 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.404180:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760773.404185:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547655 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.404194:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547655 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.404214:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547655 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.404223:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547655 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.404396:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495899648:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35110us (35467us total) trans 4294973700 rc 0/0 00000100:00100000:3.0:1789760773.404408:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6412 00000100:00100000:3.0:1789760773.404418:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757647539389 time 757689985703 state 0x2 ref 0 00000100:00100000:2.0:1789760773.407961:0:3318:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760773.408057:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f03095c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3318:1876699520313344:0@lo:400:kworker.0 00000100:00100000:3.0:1789760773.408672:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760773.408793:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495899776 00000100:00100000:0.0:1789760773.408801:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495899776 00000100:00100000:0.0:1789760773.408861:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757647539389 time 757689985703 state 0x0 ref 1 00000100:00100000:0.0:1789760773.408876:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757690246119 time 757689985703 state 0x1 ref 1 00000100:00100000:0.0:1789760773.408911:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757690246119 time 757729627814 state 0x0 ref 1 00000100:00100000:0.0:1789760773.408923:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495899776:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760773.408968:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26148864, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc809 00010000:00010000:0.0:1789760773.408978:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454765c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.408994:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760773.408998:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.409004:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454765c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.409017:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454765c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.410770:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f04291180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520313600:0@lo:400:kworker.0 00000100:00100000:1.0:1789760773.411004:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760773.411132:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789760773.411140:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f04291500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699520313472:0@lo:400:kworker.0 00000100:00100000:1.0:1789760773.411175:0:10490:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520313600 00000100:00100000:1.0:1789760773.411181:0:10490:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520313600 00000100:00100000:1.0:1789760773.411229:0:10490:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 236 00000100:00100000:1.0:1789760773.411237:0:10490:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04290700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520313600:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760773.411305:0:10490:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04290700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520313600:12345-0@lo:400:kworker.0 Request processed in 68us (300us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760773.411314:0:10490:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 236 00000100:00100000:2.0:1789760773.411433:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760773.411479:0:8003:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520313344 00000100:00100000:2.0:1789760773.411484:0:8003:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520313344 00000100:00100000:2.0:1789760773.411520:0:8003:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 512 00000100:00100000:2.0:1789760773.411528:0:8003:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3318:x1876699520313344:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760773.411564:0:8003:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3318:x1876699520313344:12345-0@lo:400:kworker.0 Request processed in 37us (3440us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760773.411569:0:8003:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 512 00000100:00100000:2.0:1789760773.411589:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760773.411625:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f04291180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520313600:0@lo:400:kworker.0 00000100:00100000:2.0:1789760773.411659:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520313472 00000100:00100000:2.0:1789760773.411663:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520313472 00000100:00100000:2.0:1789760773.411688:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 513 00000100:00100000:2.0:1789760773.411695:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699520313472:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760773.411741:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699520313472:12345-0@lo:400:kworker.0 Request processed in 34us (139us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760773.411748:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 513 00000100:00100000:2.0:1789760773.411780:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f04291500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699520313472:0@lo:400:kworker.0 00000100:00100000:2.0:1789760773.411814:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f03095c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3318:1876699520313344:0@lo:400:kworker.0 00000100:00100000:2.0:1789760773.411822:0:3318:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f03096680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:59c4e264-9b75-482b-8813-3d1dc124f384:3318:1876699520313216:0@lo:400:kworker.0 00000100:00100000:0.0:1789760773.411999:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f04293100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3317:1876699520313728:0@lo:400:kworker.0 00000100:00100000:0.0:1789760773.412029:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760773.412056:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760773.412062:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f04292a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3317:1876699520313856:0@lo:400:kworker.0 00000100:00100000:0.0:1789760773.412085:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1789760773.412111:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789760773.412116:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760773.412149:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520313728 00000100:00100000:0.0:1789760773.412153:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520313728 00000100:00100000:0.0:1789760773.412182:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 514 00000100:00100000:0.0:1789760773.412188:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1876699520313728:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760773.412222:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1876699520313728:12345-0@lo:400:kworker.0 Request processed in 34us (192us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760773.412228:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 514 00000100:00100000:0.0:1789760773.412244:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520313856 00000100:00100000:0.0:1789760773.412246:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520313856 00000100:00100000:0.0:1789760773.412267:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 237 00000100:00100000:0.0:1789760773.412273:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3317:x1876699520313856:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760773.412318:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3317:x1876699520313856:12345-0@lo:400:kworker.0 Request processed in 45us (232us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760773.412325:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 237 00000100:00080000:0.0:1789760773.412343:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789760773.412345:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760773.412360:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f04292a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3317:1876699520313856:0@lo:400:kworker.0 00000100:00100000:0.0:1789760773.412369:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f04293100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3317:1876699520313728:0@lo:400:kworker.0 00000100:00100000:2.0:1789760773.412589:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760773.412724:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520313216 00000100:00100000:1.0:1789760773.412728:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520313216 00000100:00100000:1.0:1789760773.412763:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 413 00000100:00100000:1.0:1789760773.412769:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290122cfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:59c4e264-9b75-482b-8813-3d1dc124f384+19:3318:x1876699520313216:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760773.413172:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290122cfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:59c4e264-9b75-482b-8813-3d1dc124f384+19:3318:x1876699520313216:12345-0@lo:400:kworker.0 Request processed in 402us (582us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760773.413180:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 413 00000100:00100000:2.0:1789760773.415485:0:3318:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f03096680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:59c4e264-9b75-482b-8813-3d1dc124f384:3318:1876699520313216:0@lo:400:kworker.0 00002000:00010000:0.0:1789760773.443262:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26148864 -> 26152960 00010000:00010000:0.0:1789760773.443312:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454765c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.443327:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454765c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.443336:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454765c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.443342:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760773.443348:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454765c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.443357:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454765c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.443378:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454765c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.443385:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454765c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.443478:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495899776:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34552us (34803us total) trans 4294973701 rc 0/0 00000100:00100000:0.0:1789760773.443489:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6413 00000100:00100000:0.0:1789760773.443500:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757690246119 time 757729627814 state 0x2 ref 0 00000001:00080000:1.0:1789760773.443637:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973701 is committed 00000100:00100000:1.0:1789760773.449630:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760773.449772:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495899904 00000100:00100000:0.0:1789760773.449783:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495899904 00000100:00100000:0.0:1789760773.449845:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757690246119 time 757729627814 state 0x0 ref 1 00000100:00100000:0.0:1789760773.449861:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757729888230 time 757729627814 state 0x1 ref 1 00000100:00100000:0.0:1789760773.449891:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757729888230 time 757770608199 state 0x0 ref 1 00000100:00100000:0.0:1789760773.449904:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495899904:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760773.449962:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26152960, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc809 00010000:00010000:0.0:1789760773.449972:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547663 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.449988:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760773.449992:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.449998:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547663 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.450012:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547663 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760773.513743:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26152960 -> 26157056 00010000:00010000:0.0:1789760773.513764:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547663 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.513777:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547663 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.513797:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547663 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.513812:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760773.513816:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547663 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.513825:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547663 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.513844:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547663 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.513851:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547663 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.513939:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495899904:12345-192.168.203.9@tcp:4:dd.0 Request processed in 64032us (64313us total) trans 4294973702 rc 0/0 00000100:00100000:0.0:1789760773.513949:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6414 00000100:00100000:0.0:1789760773.513961:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757729888230 time 757770608199 state 0x2 ref 0 00000001:00080000:1.0:1789760773.514156:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973702 is committed 00000100:00100000:0.0:1789760773.518375:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760773.518470:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495900032 00000100:00100000:3.0:1789760773.518478:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495900032 00000100:00100000:3.0:1789760773.518536:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757729888230 time 757770608199 state 0x0 ref 1 00000100:00100000:3.0:1789760773.518551:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757770868615 time 757770608199 state 0x1 ref 1 00000100:00100000:3.0:1789760773.518603:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757770868615 time 757839319595 state 0x0 ref 1 00000100:00100000:3.0:1789760773.518615:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495900032:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760773.518658:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26157056, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc809 00010000:00010000:3.0:1789760773.518667:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454766a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.518683:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760773.518687:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.518693:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454766a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.518707:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454766a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.546275:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973703 is committed 00002000:00010000:3.0:1789760773.546403:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26157056 -> 26161152 00010000:00010000:3.0:1789760773.546417:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454766a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.546430:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454766a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.546440:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454766a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.546447:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760773.546452:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454766a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.546462:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454766a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.546480:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454766a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.546486:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454766a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.546610:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495900032:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27992us (28238us total) trans 4294973703 rc 0/0 00000100:00100000:3.0:1789760773.546624:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6415 00000100:00100000:3.0:1789760773.546635:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757770868615 time 757839319595 state 0x2 ref 0 00000100:00100000:0.0:1789760773.551183:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760773.555792:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495900160 00000100:00100000:2.0:1789760773.555814:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495900160 00000100:00100000:2.0:1789760773.555876:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757770868615 time 757839319595 state 0x0 ref 1 00000100:00100000:2.0:1789760773.555892:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757839580011 time 757839319595 state 0x1 ref 1 00000100:00100000:2.0:1789760773.555918:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757839580011 time 757876634845 state 0x0 ref 1 00000100:00100000:2.0:1789760773.555932:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495900160:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760773.555976:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26161152, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc809 00010000:00010000:2.0:1789760773.555985:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547671 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.556002:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760773.556007:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.556012:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547671 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.556027:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547671 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760773.574190:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26161152 -> 26165248 00010000:00010000:2.0:1789760773.574210:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547671 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.574223:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547671 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760773.574227:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973704 is committed 00010000:00010000:2.0:1789760773.574232:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547671 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.574239:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760773.574245:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547671 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.574254:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547671 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.574273:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547671 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.574317:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547671 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.574399:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495900160:12345-192.168.203.9@tcp:4:dd.0 Request processed in 18465us (23278us total) trans 4294973704 rc 0/0 00000100:00100000:2.0:1789760773.574409:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6416 00000100:00100000:2.0:1789760773.574418:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757839580011 time 757876634845 state 0x2 ref 0 00000100:00100000:1.0:1789760773.577063:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760773.577121:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495900288 00000100:00100000:0.0:1789760773.577130:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495900288 00000100:00100000:0.0:1789760773.577181:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757839580011 time 757876634845 state 0x0 ref 1 00000100:00100000:0.0:1789760773.577196:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757876895261 time 757876634845 state 0x1 ref 1 00000100:00100000:0.0:1789760773.577223:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757876895261 time 757897939666 state 0x0 ref 1 00000100:00100000:0.0:1789760773.577236:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495900288:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760773.577327:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26165248, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc809 00010000:00010000:0.0:1789760773.577336:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547678 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.577353:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760773.577358:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.577365:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547678 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.577377:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547678 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760773.596641:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973705 is committed 00002000:00010000:0.0:1789760773.596720:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26165248 -> 26169344 00010000:00010000:0.0:1789760773.596736:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547678 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.596750:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547678 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.596759:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547678 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.596768:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760773.596773:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547678 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.596783:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547678 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.596813:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547678 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.596827:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547678 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.596925:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495900288:12345-192.168.203.9@tcp:4:dd.0 Request processed in 19686us (19866us total) trans 4294973705 rc 0/0 00000100:00100000:0.0:1789760773.596935:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6417 00000100:00100000:0.0:1789760773.596945:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757876895261 time 757897939666 state 0x2 ref 0 00000100:00100000:0.0:1789760773.601711:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760773.601980:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495900416 00000100:00100000:3.0:1789760773.601992:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495900416 00000100:00100000:3.0:1789760773.602058:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757876895261 time 757897939666 state 0x0 ref 1 00000100:00100000:3.0:1789760773.602076:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757898200082 time 757897939666 state 0x1 ref 1 00000100:00100000:3.0:1789760773.602114:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757898200082 time 757922830559 state 0x0 ref 1 00000100:00100000:3.0:1789760773.602128:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495900416:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760773.602188:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26169344, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc809 00010000:00010000:3.0:1789760773.603337:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454767f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.603362:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760773.603367:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.603374:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454767f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.603391:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454767f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760773.629053:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26169344 -> 26173440 00010000:00010000:0.0:1789760773.629073:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454767f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.629086:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454767f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.629096:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454767f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.629103:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760773.629107:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454767f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.629116:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454767f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.629141:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454767f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.629149:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454767f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.629240:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495900416:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27110us (27534us total) trans 4294973706 rc 0/0 00000100:00100000:0.0:1789760773.629251:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6418 00000100:00100000:0.0:1789760773.629261:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757898200082 time 757922830559 state 0x2 ref 0 00000001:00080000:1.0:1789760773.629372:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973706 is committed 00000100:00100000:3.0:1789760773.638170:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760773.638381:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495900544 00000100:00100000:2.0:1789760773.638391:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495900544 00000100:00100000:2.0:1789760773.638526:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757898200082 time 757922830559 state 0x0 ref 1 00000100:00100000:2.0:1789760773.638544:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757923090975 time 757922830559 state 0x1 ref 1 00000100:00100000:2.0:1789760773.638606:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757923090975 time 757959322600 state 0x0 ref 1 00000100:00100000:2.0:1789760773.638619:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495900544:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760773.638668:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26173440, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc809 00010000:00010000:2.0:1789760773.638678:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547686 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.638694:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760773.638698:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.638705:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547686 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.638720:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547686 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.675551:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973707 is committed 00002000:00010000:0.0:1789760773.675705:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26173440 -> 26177536 00002000:00010000:0.0:1789760773.675709:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760772 -> 1789760773 00002000:00010000:0.0:1789760773.675712:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760772 -> 1789760773 00010000:00010000:0.0:1789760773.675725:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547686 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.675738:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547686 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.675747:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547686 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.675755:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760773.675761:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547686 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.675770:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547686 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.675808:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547686 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.675816:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547686 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.675914:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495900544:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37288us (37745us total) trans 4294973707 rc 0/0 00000100:00100000:0.0:1789760773.675924:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6419 00000100:00100000:0.0:1789760773.675935:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757923090975 time 757959322600 state 0x2 ref 0 00000100:00100000:2.0:1789760773.686578:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760773.688743:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495900672 00000100:00100000:0.0:1789760773.688755:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495900672 00000100:00100000:0.0:1789760773.688827:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757923090975 time 757959322600 state 0x0 ref 1 00000100:00100000:0.0:1789760773.688842:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757959583016 time 757959322600 state 0x1 ref 1 00000100:00100000:0.0:1789760773.688865:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757959583016 time 758009581333 state 0x0 ref 1 00000100:00100000:0.0:1789760773.688881:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495900672:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760773.688930:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26177536, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc809 00010000:00010000:0.0:1789760773.688939:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454768d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.688957:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760773.688962:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.688969:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454768d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.688985:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454768d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.719236:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973708 is committed 00002000:00010000:0.0:1789760773.719277:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26177536 -> 26181632 00010000:00010000:0.0:1789760773.719396:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454768d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.719409:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454768d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.719418:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454768d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.719426:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760773.719431:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454768d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.719440:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454768d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.719455:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454768d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.719461:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454768d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.719585:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495900672:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30704us (33012us total) trans 4294973708 rc 0/0 00000100:00100000:0.0:1789760773.719592:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6420 00000100:00100000:0.0:1789760773.719600:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757959583016 time 758009581333 state 0x2 ref 0 00000100:00100000:2.0:1789760773.726606:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760773.728489:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495900800 00000100:00100000:0.0:1789760773.728504:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495900800 00000100:00100000:0.0:1789760773.728569:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 757959583016 time 758009581333 state 0x0 ref 1 00000100:00100000:0.0:1789760773.728585:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758009841749 time 758009581333 state 0x1 ref 1 00000100:00100000:0.0:1789760773.728680:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758009841749 time 758049395375 state 0x0 ref 1 00000100:00100000:0.0:1789760773.728696:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495900800:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760773.728747:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26181632, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc809 00010000:00010000:0.0:1789760773.728756:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547694 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.728773:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760773.728778:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.728784:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547694 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.728817:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547694 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.765680:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973709 is committed 00002000:00010000:0.0:1789760773.765750:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26181632 -> 26185728 00010000:00010000:0.0:1789760773.765765:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547694 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.765786:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547694 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.765796:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547694 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.765813:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760773.765818:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547694 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.765828:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547694 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.765852:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547694 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.765860:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547694 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.765953:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495900800:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37257us (39354us total) trans 4294973709 rc 0/0 00000100:00100000:0.0:1789760773.765961:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6421 00000100:00100000:0.0:1789760773.765976:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758009841749 time 758049395375 state 0x2 ref 0 00000100:00100000:3.0:1789760773.768371:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760773.768484:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495900928 00000100:00100000:0.0:1789760773.768496:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495900928 00000100:00100000:0.0:1789760773.768554:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758009841749 time 758049395375 state 0x0 ref 1 00000100:00100000:0.0:1789760773.768569:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758049655791 time 758049395375 state 0x1 ref 1 00000100:00100000:0.0:1789760773.768594:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758049655791 time 758089310724 state 0x0 ref 1 00000100:00100000:0.0:1789760773.768605:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fdca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495900928:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760773.768660:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26185728, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc809 00010000:00010000:0.0:1789760773.768668:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454769b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.768684:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760773.768688:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.768694:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454769b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.768707:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454769b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760773.812647:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973710 is committed 00002000:00010000:0.0:1789760773.812938:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26185728 -> 26189824 00010000:00010000:0.0:1789760773.812956:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454769b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.812976:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454769b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.812986:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454769b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.812994:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760773.812999:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454769b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.813008:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454769b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.813029:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454769b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.813036:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454769b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.813132:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fdca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495900928:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44524us (44765us total) trans 4294973710 rc 0/0 00000100:00100000:0.0:1789760773.813147:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6422 00000100:00100000:0.0:1789760773.813156:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758049655791 time 758089310724 state 0x2 ref 0 00000100:00100000:2.0:1789760773.816540:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760773.817058:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495901056 00000100:00100000:0.0:1789760773.817067:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495901056 00000100:00100000:0.0:1789760773.817120:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758049655791 time 758089310724 state 0x0 ref 1 00000100:00100000:0.0:1789760773.817251:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758089571140 time 758089310724 state 0x1 ref 1 00000100:00100000:0.0:1789760773.817403:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758089571140 time 758138117984 state 0x0 ref 1 00000100:00100000:0.0:1789760773.817417:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495901056:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760773.817467:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26189824, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc809 00010000:00010000:0.0:1789760773.817477:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545476a2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.817493:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760773.817497:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.817503:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545476a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.817518:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545476a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.847618:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973711 is committed 00002000:00010000:1.0:1789760773.847707:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26189824 -> 26193920 00010000:00010000:1.0:1789760773.847721:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545476a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.847734:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545476a2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.847743:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545476a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.847750:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760773.847756:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545476a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.847764:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545476a2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.847846:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545476a2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.847855:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545476a2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.847984:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495901056:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30565us (31447us total) trans 4294973711 rc 0/0 00000100:00100000:1.0:1789760773.847995:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6423 00000100:00100000:1.0:1789760773.848006:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758089571140 time 758138117984 state 0x2 ref 0 00000100:00100000:2.0:1789760773.852617:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760773.852688:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495901184 00000100:00100000:0.0:1789760773.852697:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495901184 00000100:00100000:0.0:1789760773.852753:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758089571140 time 758138117984 state 0x0 ref 1 00000100:00100000:0.0:1789760773.852767:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758138378400 time 758138117984 state 0x1 ref 1 00000100:00100000:0.0:1789760773.852797:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758138378400 time 758173514081 state 0x0 ref 1 00000100:00100000:0.0:1789760773.852820:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495901184:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760773.852879:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26193920, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc809 00010000:00010000:0.0:1789760773.852893:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545476a9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.852909:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760773.852914:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.852922:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545476a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.852936:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545476a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760773.891563:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26193920 -> 26198016 00000001:00080000:3.0:1789760773.891573:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973712 is committed 00010000:00010000:1.0:1789760773.891583:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545476a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.891596:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545476a9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.891606:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545476a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.891619:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760773.891624:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545476a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.891634:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545476a9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.891653:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545476a9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.891658:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545476a9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.891846:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495901184:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38957us (39309us total) trans 4294973712 rc 0/0 00000100:00100000:1.0:1789760773.891857:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6424 00000100:00100000:1.0:1789760773.891867:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758138378400 time 758173514081 state 0x2 ref 0 00000100:00100000:2.0:1789760773.899076:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760773.899131:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495901312 00000100:00100000:0.0:1789760773.899142:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495901312 00000100:00100000:0.0:1789760773.899198:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758138378400 time 758173514081 state 0x0 ref 1 00000100:00100000:0.0:1789760773.899214:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758173774497 time 758173514081 state 0x1 ref 1 00000100:00100000:0.0:1789760773.899341:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758173774497 time 758220056726 state 0x0 ref 1 00000100:00100000:0.0:1789760773.899354:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495901312:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760773.899414:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26198016, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc809 00010000:00010000:0.0:1789760773.899424:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545476b0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.899441:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760773.899446:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.899456:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545476b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.899471:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545476b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.918993:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f04292a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520313984:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760773.919060:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760773.919394:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520313984 00000100:00100000:1.0:1789760773.919402:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520313984 00000100:00100000:1.0:1789760773.919447:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 212 00000100:00100000:1.0:1789760773.919457:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520313984:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760773.919488:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760773.919545:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520313984:12345-0@lo:13:osp-pre-1-0.0 Request processed in 90us (484us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760773.919556:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 212 00000100:00100000:2.0:1789760773.919692:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f04292a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520313984:0@lo:13:osp-pre-1-0.0 00000001:00080000:1.0:1789760773.930167:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973713 is committed 00002000:00010000:0.0:1789760773.930296:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26198016 -> 26202112 00010000:00010000:0.0:1789760773.930311:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545476b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.930324:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545476b0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.930332:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545476b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.930339:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760773.930344:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545476b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.930354:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545476b0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.930386:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545476b0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.930392:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545476b0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.930522:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495901312:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31165us (31740us total) trans 4294973713 rc 0/0 00000100:00100000:0.0:1789760773.930533:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6425 00000100:00100000:0.0:1789760773.930544:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758173774497 time 758220056726 state 0x2 ref 0 00000100:00100000:2.0:1789760773.938988:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760773.939083:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495901440 00000100:00100000:1.0:1789760773.939093:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495901440 00000100:00100000:1.0:1789760773.939144:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758173774497 time 758220056726 state 0x0 ref 1 00000100:00100000:1.0:1789760773.939160:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758220317142 time 758220056726 state 0x1 ref 1 00000100:00100000:1.0:1789760773.939197:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758220317142 time 758259913307 state 0x0 ref 1 00000100:00100000:1.0:1789760773.939211:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03096680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495901440:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760773.939261:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26202112, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc809 00010000:00010000:1.0:1789760773.939269:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476b7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.939505:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760773.939510:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.939518:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.939533:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760773.970954:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26202112 -> 26206208 00010000:00010000:1.0:1789760773.970974:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.970990:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476b7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.970998:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.971004:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760773.971009:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.971017:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476b7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.971035:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476b7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.971043:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476b7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.971130:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03096680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495901440:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31918us (32146us total) trans 4294973714 rc 0/0 00000100:00100000:1.0:1789760773.971140:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6426 00000100:00100000:1.0:1789760773.971151:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758220317142 time 758259913307 state 0x2 ref 0 00000001:00080000:1.0:1789760773.971190:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973714 is committed 00000100:00100000:1.0:1789760773.974530:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760773.974608:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495901568 00000100:00100000:0.0:1789760773.974617:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495901568 00000100:00100000:0.0:1789760773.974670:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758220317142 time 758259913307 state 0x0 ref 1 00000100:00100000:0.0:1789760773.974685:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758260173723 time 758259913307 state 0x1 ref 1 00000100:00100000:0.0:1789760773.974709:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758260173723 time 758295425899 state 0x0 ref 1 00000100:00100000:0.0:1789760773.974721:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04291500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495901568:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760773.974769:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26206208, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc809 00010000:00010000:0.0:1789760773.974792:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545476be lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.974818:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760773.974823:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.974831:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545476be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.974845:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545476be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.983267:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760774.005138:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973715 is committed 00002000:00010000:2.0:1789760774.005322:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26206208 -> 26210304 00010000:00010000:2.0:1789760774.005337:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545476be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.005351:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545476be lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.005365:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545476be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.005373:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760774.005378:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545476be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.005387:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545476be lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.005407:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545476be lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.005416:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545476be lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.005515:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04291500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495901568:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30790us (30989us total) trans 4294973715 rc 0/0 00000100:00100000:2.0:1789760774.005525:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6427 00000100:00100000:2.0:1789760774.005534:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758260173723 time 758295425899 state 0x2 ref 0 00000100:00100000:3.0:1789760774.009472:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760774.009624:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495901696 00000100:00100000:0.0:1789760774.009633:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495901696 00000100:00100000:0.0:1789760774.009691:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758260173723 time 758295425899 state 0x0 ref 1 00000100:00100000:0.0:1789760774.009708:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758295686315 time 758295425899 state 0x1 ref 1 00000100:00100000:0.0:1789760774.009770:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758295686315 time 758330486442 state 0x0 ref 1 00000100:00100000:0.0:1789760774.009792:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495901696:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760774.009853:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26210304, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc809 00010000:00010000:0.0:1789760774.009862:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545476c5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.009877:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760774.009882:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.009889:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545476c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.009908:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545476c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.034947:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973716 is committed 00002000:00010000:1.0:1789760774.035203:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26210304 -> 26214400 00010000:00010000:1.0:1789760774.035219:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545476c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.035232:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545476c5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.035242:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545476c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.035249:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760774.035253:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545476c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.035263:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545476c5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.035305:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545476c5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.035312:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545476c5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.035398:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495901696:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25604us (25928us total) trans 4294973716 rc 0/0 00000100:00100000:1.0:1789760774.035407:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6428 00000100:00100000:1.0:1789760774.035419:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758295686315 time 758330486442 state 0x2 ref 0 00000100:00100000:3.0:1789760774.038587:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760774.038910:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495901824 00000100:00100000:1.0:1789760774.038919:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495901824 00000100:00100000:1.0:1789760774.038973:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758295686315 time 758330486442 state 0x0 ref 1 00000100:00100000:1.0:1789760774.038989:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758330746858 time 758330486442 state 0x1 ref 1 00000100:00100000:1.0:1789760774.039019:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758330746858 time 758359735671 state 0x0 ref 1 00000100:00100000:1.0:1789760774.039030:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495901824:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760774.039078:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26214400, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc809 00010000:00010000:1.0:1789760774.039087:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476cc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.039102:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760774.039106:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.039112:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.039230:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.070117:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973717 is committed 00002000:00010000:1.0:1789760774.070261:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26214400 -> 26218496 00002000:00010000:1.0:1789760774.070266:0:13055:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 51209 -> 53257 00010000:00010000:1.0:1789760774.070302:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.070320:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476cc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.070329:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.070335:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760774.070341:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.070349:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476cc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.070371:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476cc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.070378:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476cc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.070468:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495901824:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31436us (31886us total) trans 4294973717 rc 0/0 00000100:00100000:1.0:1789760774.070479:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6429 00000100:00100000:1.0:1789760774.070489:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758330746858 time 758359735671 state 0x2 ref 0 00000100:00100000:2.0:1789760774.077960:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760774.078034:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495901952 00000100:00100000:0.0:1789760774.078043:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495901952 00000100:00100000:0.0:1789760774.078098:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758330746858 time 758359735671 state 0x0 ref 1 00000100:00100000:0.0:1789760774.078114:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758359996087 time 758359735671 state 0x1 ref 1 00000100:00100000:0.0:1789760774.078143:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758359996087 time 758398860037 state 0x0 ref 1 00000100:00100000:0.0:1789760774.078156:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495901952:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760774.078205:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26218496, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd009 00010000:00010000:0.0:1789760774.078214:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545476d3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.078230:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760774.078234:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.078241:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545476d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.078255:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545476d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760774.114714:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26218496 -> 26222592 00010000:00010000:1.0:1789760774.114739:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545476d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.114753:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545476d3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.114762:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545476d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.114770:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760774.114786:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545476d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.114805:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545476d3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.114826:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545476d3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.114835:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545476d3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.115043:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495901952:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36884us (37088us total) trans 4294973718 rc 0/0 00000100:00100000:1.0:1789760774.115055:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6430 00000100:00100000:1.0:1789760774.115064:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758359996087 time 758398860037 state 0x2 ref 0 00000001:00080000:3.0:1789760774.115409:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973718 is committed 00000100:00100000:2.0:1789760774.118767:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760774.118998:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495902080 00000100:00100000:1.0:1789760774.119007:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495902080 00000100:00100000:1.0:1789760774.119055:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758359996087 time 758398860037 state 0x0 ref 1 00000100:00100000:1.0:1789760774.119070:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758399120453 time 758398860037 state 0x1 ref 1 00000100:00100000:1.0:1789760774.119096:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758399120453 time 758439813135 state 0x0 ref 1 00000100:00100000:1.0:1789760774.119106:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495902080:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760774.119202:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26222592, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd009 00010000:00010000:1.0:1789760774.119211:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476da lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.119226:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760774.119230:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.119236:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.119250:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.152481:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973719 is committed 00002000:00010000:1.0:1789760774.152703:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26222592 -> 26226688 00010000:00010000:1.0:1789760774.152719:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.152733:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476da lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.152742:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.152751:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760774.152757:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.152766:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476da lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.152803:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476da lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.152812:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476da lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.152905:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495902080:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33795us (34141us total) trans 4294973719 rc 0/0 00000100:00100000:1.0:1789760774.152916:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6431 00000100:00100000:1.0:1789760774.152925:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758399120453 time 758439813135 state 0x2 ref 0 00000100:00100000:2.0:1789760774.156956:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760774.157198:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495902208 00000100:00100000:1.0:1789760774.157208:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495902208 00000100:00100000:1.0:1789760774.157265:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758399120453 time 758439813135 state 0x0 ref 1 00000100:00100000:1.0:1789760774.157379:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758440073551 time 758439813135 state 0x1 ref 1 00000100:00100000:2.0:1789760774.157498:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758440073551 time 758478211806 state 0x0 ref 1 00000100:00100000:2.0:1789760774.157512:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495902208:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760774.157563:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26226688, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd009 00010000:00010000:2.0:1789760774.157572:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545476e1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.157590:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760774.157594:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.157601:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545476e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.157615:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545476e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760774.186267:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26226688 -> 26230784 00000001:00080000:3.0:1789760774.186313:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973720 is committed 00010000:00010000:1.0:1789760774.186428:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545476e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.186442:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545476e1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.186451:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545476e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.186458:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760774.186463:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545476e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.186471:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545476e1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.186491:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545476e1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.186500:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545476e1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.186664:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495902208:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29149us (29713us total) trans 4294973720 rc 0/0 00000100:00100000:2.0:1789760774.186676:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6432 00000100:00100000:2.0:1789760774.186686:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758440073551 time 758478211806 state 0x2 ref 0 00000100:00100000:3.0:1789760774.190588:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760774.190663:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495902336 00000100:00100000:1.0:1789760774.190682:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495902336 00000100:00100000:1.0:1789760774.190738:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758440073551 time 758478211806 state 0x0 ref 1 00000100:00100000:1.0:1789760774.190757:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758478472222 time 758478211806 state 0x1 ref 1 00000100:00100000:1.0:1789760774.190785:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758478472222 time 758511501860 state 0x0 ref 1 00000100:00100000:1.0:1789760774.190807:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495902336:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760774.190854:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26230784, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd009 00010000:00010000:1.0:1789760774.190863:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476e8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.190879:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760774.190884:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.190890:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.190904:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.227995:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973721 is committed 00002000:00010000:1.0:1789760774.228071:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26230784 -> 26234880 00010000:00010000:1.0:1789760774.228088:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.228101:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476e8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.228109:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.228121:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760774.228127:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.228135:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476e8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.228154:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476e8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.228161:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476e8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.228401:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495902336:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37600us (37816us total) trans 4294973721 rc 0/0 00000100:00100000:1.0:1789760774.228413:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6433 00000100:00100000:1.0:1789760774.228423:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758478472222 time 758511501860 state 0x2 ref 0 00000100:00100000:3.0:1789760774.231596:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760774.232274:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495902464 00000100:00100000:1.0:1789760774.232369:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495902464 00000100:00100000:1.0:1789760774.232428:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758478472222 time 758511501860 state 0x0 ref 1 00000100:00100000:1.0:1789760774.232444:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758511762276 time 758511501860 state 0x1 ref 1 00000100:00100000:1.0:1789760774.232477:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758511762276 time 758553194335 state 0x0 ref 1 00000100:00100000:1.0:1789760774.232492:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495902464:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760774.232538:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26234880, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd009 00010000:00010000:1.0:1789760774.232547:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545476ef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.232562:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760774.232567:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.232573:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545476ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.232587:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545476ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.258071:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973722 is committed 00002000:00010000:0.0:1789760774.258228:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26234880 -> 26238976 00010000:00010000:0.0:1789760774.258243:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545476ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.258256:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545476ef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.258266:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545476ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.258273:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760774.258306:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545476ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.258316:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545476ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.258336:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545476ef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.258344:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545476ef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.258471:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495902464:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25976us (26879us total) trans 4294973722 rc 0/0 00000100:00100000:0.0:1789760774.258481:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6434 00000100:00100000:0.0:1789760774.258492:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758511762276 time 758553194335 state 0x2 ref 0 00000100:00100000:2.0:1789760774.265984:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760774.267467:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495902592 00000100:00100000:0.0:1789760774.267479:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495902592 00000100:00100000:0.0:1789760774.267587:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758511762276 time 758553194335 state 0x0 ref 1 00000100:00100000:0.0:1789760774.267602:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758553454751 time 758553194335 state 0x1 ref 1 00000100:00100000:0.0:1789760774.267677:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758553454751 time 758588393682 state 0x0 ref 1 00000100:00100000:0.0:1789760774.267691:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495902592:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760774.267735:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26238976, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd009 00010000:00010000:0.0:1789760774.267744:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545476f6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.267759:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760774.267763:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.267769:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545476f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.267790:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545476f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.301874:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973723 is committed 00002000:00010000:0.0:1789760774.301984:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26238976 -> 26243072 00010000:00010000:0.0:1789760774.301998:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545476f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.302012:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545476f6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.302022:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545476f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.302030:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760774.302035:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545476f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.302044:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545476f6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.302065:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545476f6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.302073:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545476f6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.302265:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495902592:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34571us (36287us total) trans 4294973723 rc 0/0 00000100:00100000:0.0:1789760774.302277:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6435 00000100:00100000:0.0:1789760774.302326:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758553454751 time 758588393682 state 0x2 ref 0 00000100:00100000:2.0:1789760774.306695:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760774.306806:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495902720 00000100:00100000:1.0:1789760774.306815:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495902720 00000100:00100000:1.0:1789760774.306871:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758553454751 time 758588393682 state 0x0 ref 1 00000100:00100000:1.0:1789760774.306886:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758588654098 time 758588393682 state 0x1 ref 1 00000100:00100000:1.0:1789760774.306948:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758588654098 time 758627664211 state 0x0 ref 1 00000100:00100000:1.0:1789760774.306961:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495902720:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760774.307008:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26243072, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd009 00010000:00010000:1.0:1789760774.307017:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476fd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.307033:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760774.307038:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.307044:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.307059:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.338742:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973724 is committed 00002000:00010000:1.0:1789760774.338905:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26243072 -> 26247168 00010000:00010000:1.0:1789760774.338921:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.338933:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476fd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.338942:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.338949:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760774.338954:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.338963:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476fd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.338988:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476fd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.338996:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545476fd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.339089:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495902720:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32125us (32398us total) trans 4294973724 rc 0/0 00000100:00100000:1.0:1789760774.339100:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6436 00000100:00100000:1.0:1789760774.339110:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758588654098 time 758627664211 state 0x2 ref 0 00000100:00100000:3.0:1789760774.342939:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760774.345007:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495902848 00000100:00100000:0.0:1789760774.345021:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495902848 00000100:00100000:0.0:1789760774.345078:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758588654098 time 758627664211 state 0x0 ref 1 00000100:00100000:0.0:1789760774.345095:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758627924627 time 758627664211 state 0x1 ref 1 00000100:00100000:0.0:1789760774.345128:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758627924627 time 758665845584 state 0x0 ref 1 00000100:00100000:0.0:1789760774.345143:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495902848:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760774.345189:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26247168, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd009 00010000:00010000:0.0:1789760774.345198:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547704 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.345214:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760774.345218:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.345226:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547704 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.345243:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547704 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.381091:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973725 is committed 00002000:00010000:0.0:1789760774.381211:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26247168 -> 26251264 00010000:00010000:0.0:1789760774.381227:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547704 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.381239:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547704 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.381253:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547704 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.381262:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760774.381266:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547704 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.381276:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547704 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.381328:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547704 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.381336:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547704 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.381493:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495902848:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36348us (38559us total) trans 4294973725 rc 0/0 00000100:00100000:0.0:1789760774.381503:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6437 00000100:00100000:0.0:1789760774.381515:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758627924627 time 758665845584 state 0x2 ref 0 00000100:00100000:2.0:1789760774.386465:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760774.386653:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495902976 00000100:00100000:0.0:1789760774.386663:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495902976 00000100:00100000:0.0:1789760774.386712:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758627924627 time 758665845584 state 0x0 ref 1 00000100:00100000:0.0:1789760774.386727:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758666106000 time 758665845584 state 0x1 ref 1 00000100:00100000:0.0:1789760774.386756:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758666106000 time 758707474097 state 0x0 ref 1 00000100:00100000:0.0:1789760774.386767:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495902976:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760774.386822:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26251264, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd009 00010000:00010000:0.0:1789760774.386831:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454770b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.386847:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760774.386851:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.386857:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454770b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.386869:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454770b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760774.434354:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26251264 -> 26255360 00010000:00010000:0.0:1789760774.434375:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454770b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.434389:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454770b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.434399:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454770b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.434407:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760774.434412:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454770b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.434421:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454770b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.434442:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454770b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.434450:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454770b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.434534:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495902976:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47763us (48075us total) trans 4294973726 rc 0/0 00000100:00100000:0.0:1789760774.434545:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6438 00000100:00100000:0.0:1789760774.434554:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758666106000 time 758707474097 state 0x2 ref 0 00000001:00080000:1.0:1789760774.435327:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973726 is committed 00000100:00100000:2.0:1789760774.439552:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760774.439885:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495903104 00000100:00100000:1.0:1789760774.439895:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495903104 00000100:00100000:1.0:1789760774.439948:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758666106000 time 758707474097 state 0x0 ref 1 00000100:00100000:1.0:1789760774.439964:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758707734513 time 758707474097 state 0x1 ref 1 00000100:00100000:1.0:1789760774.440034:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758707734513 time 758760750473 state 0x0 ref 1 00000100:00100000:1.0:1789760774.440047:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495903104:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760774.440139:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26255360, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd009 00010000:00010000:1.0:1789760774.440150:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547712 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.440171:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760774.440175:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.440181:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547712 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.440194:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547712 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.493153:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973727 is committed 00002000:00010000:0.0:1789760774.493234:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26255360 -> 26259456 00010000:00010000:0.0:1789760774.493249:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547712 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.493261:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547712 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.493270:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547712 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.493361:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760774.493366:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547712 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.493374:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547712 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.493392:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547712 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.493400:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547712 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.493720:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495903104:12345-192.168.203.9@tcp:4:dd.0 Request processed in 53669us (54170us total) trans 4294973727 rc 0/0 00000100:00100000:1.0:1789760774.493731:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6439 00000100:00100000:1.0:1789760774.493741:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758707734513 time 758760750473 state 0x2 ref 0 00000100:00100000:2.0:1789760774.500558:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760774.502867:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495903232 00000100:00100000:0.0:1789760774.502880:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495903232 00000100:00100000:1.0:1789760774.502997:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758707734513 time 758760750473 state 0x0 ref 1 00000100:00100000:1.0:1789760774.503013:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758761010889 time 758760750473 state 0x1 ref 1 00000100:00100000:1.0:1789760774.503053:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758761010889 time 758823770154 state 0x0 ref 1 00000100:00100000:1.0:1789760774.503066:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495903232:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760774.503113:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26259456, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd009 00010000:00010000:1.0:1789760774.503122:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547719 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.503142:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760774.503146:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.503151:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547719 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.503164:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547719 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.549961:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973728 is committed 00002000:00010000:1.0:1789760774.550055:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26259456 -> 26263552 00010000:00010000:1.0:1789760774.550070:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547719 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.550084:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547719 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.550092:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547719 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.550099:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760774.550104:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547719 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.550112:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547719 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.550130:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547719 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.550137:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547719 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.550227:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495903232:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47159us (49672us total) trans 4294973728 rc 0/0 00000100:00100000:1.0:1789760774.550237:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6440 00000100:00100000:1.0:1789760774.550247:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758761010889 time 758823770154 state 0x2 ref 0 00000100:00100000:0.0:1789760774.555805:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760774.555891:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495903360 00000100:00100000:1.0:1789760774.555901:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495903360 00000100:00100000:1.0:1789760774.555954:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758761010889 time 758823770154 state 0x0 ref 1 00000100:00100000:1.0:1789760774.555969:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758824030570 time 758823770154 state 0x1 ref 1 00000100:00100000:1.0:1789760774.556002:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758824030570 time 758876718992 state 0x0 ref 1 00000100:00100000:1.0:1789760774.556013:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495903360:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760774.556059:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26263552, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd009 00010000:00010000:1.0:1789760774.556068:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547720 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.556084:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760774.556088:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.556094:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547720 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.556113:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547720 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.600276:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973729 is committed 00002000:00010000:0.0:1789760774.600469:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26263552 -> 26267648 00010000:00010000:0.0:1789760774.600487:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547720 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.600500:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547720 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.600510:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547720 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.600524:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760774.600530:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547720 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.600539:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547720 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.600560:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547720 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.600568:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547720 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.600712:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495903360:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44695us (44923us total) trans 4294973729 rc 0/0 00000100:00100000:0.0:1789760774.600724:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6441 00000100:00100000:0.0:1789760774.600735:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758824030570 time 758876718992 state 0x2 ref 0 00000100:00100000:2.0:1789760774.609569:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760774.609640:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495903488 00000100:00100000:1.0:1789760774.609650:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495903488 00000100:00100000:1.0:1789760774.609703:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758824030570 time 758876718992 state 0x0 ref 1 00000100:00100000:1.0:1789760774.609717:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758876979408 time 758876718992 state 0x1 ref 1 00000100:00100000:1.0:1789760774.609746:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758876979408 time 758930462506 state 0x0 ref 1 00000100:00100000:1.0:1789760774.609758:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495903488:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760774.609810:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26267648, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd009 00010000:00010000:1.0:1789760774.609820:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547727 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.609834:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760774.609838:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.609845:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547727 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.609859:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547727 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.633128:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973730 is committed 00002000:00010000:0.0:1789760774.633251:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26267648 -> 26271744 00010000:00010000:0.0:1789760774.633265:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547727 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.633304:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547727 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.633319:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547727 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.633326:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760774.633330:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547727 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.633345:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547727 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.633363:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547727 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.633381:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547727 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.633470:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495903488:12345-192.168.203.9@tcp:4:dd.0 Request processed in 23708us (23905us total) trans 4294973730 rc 0/0 00000100:00100000:0.0:1789760774.633481:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6442 00000100:00100000:0.0:1789760774.633490:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758876979408 time 758930462506 state 0x2 ref 0 00000100:00100000:2.0:1789760774.639465:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760774.639641:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495903616 00000100:00100000:1.0:1789760774.639651:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495903616 00000100:00100000:1.0:1789760774.639707:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758876979408 time 758930462506 state 0x0 ref 1 00000100:00100000:1.0:1789760774.639721:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758930722922 time 758930462506 state 0x1 ref 1 00000100:00100000:1.0:1789760774.639751:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758930722922 time 758960467997 state 0x0 ref 1 00000100:00100000:1.0:1789760774.639762:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495903616:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760774.640431:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26271744, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd009 00010000:00010000:1.0:1789760774.640441:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454772e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.640456:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760774.640460:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.640466:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454772e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.640478:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454772e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.642886:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760774.642944:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495903872 00000100:00100000:0.0:1789760774.642955:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495903872 00000100:00100000:0.0:1789760774.643089:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 515 00000100:00100000:0.0:1789760774.643101:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:17022965-1e4f-46fb-915c-61730096b8e2+8:2357:x1876699495903872:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:0.0:1789760774.643183:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:17022965-1e4f-46fb-915c-61730096b8e2+8:2357:x1876699495903872:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 84us (304us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760774.643191:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 515 00000100:00100000:0.0:1789760774.644216:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760774.644348:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00080000:1.0:1789760774.645489:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760774.646493:0:12710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495904000 00000100:00100000:2.0:1789760774.646504:0:12710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495904000 00000100:00100000:2.0:1789760774.646664:0:12710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 516 00000100:00100000:2.0:1789760774.646675:0:12710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2357:x1876699495904000:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:2.0:1789760774.646779:0:12710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2357:x1876699495904000:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 97us (2421us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760774.646787:0:12710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 516 00000100:00080000:2.0:1789760774.646822:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760774.647233:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495903744 00000100:00100000:3.0:1789760774.647239:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495903744 00000100:00100000:3.0:1789760774.647345:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 414 00000100:00100000:3.0:1789760774.647353:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290122cd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2357:x1876699495903744:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:3.0:1789760774.647391:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290122cd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2357:x1876699495903744:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 39us (3178us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760774.647397:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 414 00000001:00080000:0.0:1789760774.690573:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973731 is committed 00002000:00010000:2.0:1789760774.691007:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26271744 -> 26275840 00002000:00010000:2.0:1789760774.691013:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760773 -> 1789760774 00002000:00010000:2.0:1789760774.691015:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760773 -> 1789760774 00010000:00010000:2.0:1789760774.691027:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454772e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.691040:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454772e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.691049:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454772e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.691057:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760774.691062:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454772e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.691070:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454772e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.691093:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454772e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.691102:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x493674835454772e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.691507:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495903616:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51740us (52046us total) trans 4294973731 rc 0/0 00000100:00100000:2.0:1789760774.691519:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6443 00000100:00100000:2.0:1789760774.691529:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758930722922 time 758960467997 state 0x2 ref 0 00000100:00100000:1.0:1789760774.694630:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760774.694722:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495904128 00000100:00100000:0.0:1789760774.694731:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495904128 00000100:00100000:0.0:1789760774.694787:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758930722922 time 758960467997 state 0x0 ref 1 00000100:00100000:0.0:1789760774.694813:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758960728413 time 758960467997 state 0x1 ref 1 00000100:00100000:0.0:1789760774.694879:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758960728413 time 759015595312 state 0x0 ref 1 00000100:00100000:0.0:1789760774.694892:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04291180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495904128:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760774.694939:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26275840, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd009 00010000:00010000:0.0:1789760774.694949:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547735 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.694965:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760774.694970:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.694976:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547735 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.694989:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547735 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.734776:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973732 is committed 00002000:00010000:0.0:1789760774.734825:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26275840 -> 26279936 00010000:00010000:0.0:1789760774.734841:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547735 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.734854:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547735 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.734862:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547735 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.734869:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94aa00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760774.734874:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547735 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.734882:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547735 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.734900:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547735 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.734912:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547735 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.735159:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04291180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495904128:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40264us (40531us total) trans 4294973732 rc 0/0 00000100:00100000:0.0:1789760774.735174:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6444 00000100:00100000:0.0:1789760774.735184:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758960728413 time 759015595312 state 0x2 ref 0 00000100:00100000:1.0:1789760774.738174:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760774.738500:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495904256 00000100:00100000:0.0:1789760774.738582:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495904256 00000100:00100000:0.0:1789760774.738637:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 758960728413 time 759015595312 state 0x0 ref 1 00000100:00100000:0.0:1789760774.738652:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759015855728 time 759015595312 state 0x1 ref 1 00000100:00100000:0.0:1789760774.738723:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759015855728 time 759059439540 state 0x0 ref 1 00000100:00100000:0.0:1789760774.738736:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04292a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495904256:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760774.738783:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26279936, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd009 00010000:00010000:0.0:1789760774.738804:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454773c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.738821:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760774.738825:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.738831:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454773c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.738843:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454773c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.775002:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973733 is committed 00002000:00010000:1.0:1789760774.775390:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26279936 -> 26284032 00010000:00010000:1.0:1789760774.775405:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454773c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.775424:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454773c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.775434:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454773c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.775442:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ae00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760774.775447:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454773c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.775457:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454773c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.775476:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454773c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.775485:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454773c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.775580:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04292a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495904256:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36841us (37409us total) trans 4294973733 rc 0/0 00000100:00100000:1.0:1789760774.775591:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6445 00000100:00100000:1.0:1789760774.775601:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759015855728 time 759059439540 state 0x2 ref 0 00000100:00100000:3.0:1789760774.780956:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760774.781107:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495904384 00000100:00100000:2.0:1789760774.781126:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495904384 00000100:00100000:2.0:1789760774.781180:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759015855728 time 759059439540 state 0x0 ref 1 00000100:00100000:2.0:1789760774.781196:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759059699956 time 759059439540 state 0x1 ref 1 00000100:00100000:2.0:1789760774.781228:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759059699956 time 759101945210 state 0x0 ref 1 00000100:00100000:2.0:1789760774.781241:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495904384:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760774.781324:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26284032, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd009 00010000:00010000:2.0:1789760774.781334:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547743 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.781356:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:2.0:1789760774.781366:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.781374:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547743 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.781387:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547743 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760774.806214:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26284032 -> 26288128 00010000:00010000:3.0:1789760774.806233:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547743 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.806247:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547743 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.806257:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547743 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.806265:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760774.806271:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547743 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.806299:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547743 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.806318:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547743 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.806327:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547743 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.806421:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495904384:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25178us (25470us total) trans 4294973734 rc 0/0 00000100:00100000:3.0:1789760774.806432:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6446 00000100:00100000:3.0:1789760774.806441:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759059699956 time 759101945210 state 0x2 ref 0 00000001:00080000:2.0:1789760774.806546:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973734 is committed 00000100:00100000:3.0:1789760774.814919:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760774.815498:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495904512 00000100:00100000:0.0:1789760774.815508:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495904512 00000100:00100000:0.0:1789760774.815563:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759059699956 time 759101945210 state 0x0 ref 1 00000100:00100000:0.0:1789760774.815578:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759102205626 time 759101945210 state 0x1 ref 1 00000100:00100000:0.0:1789760774.815609:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759102205626 time 759136326085 state 0x0 ref 1 00000100:00100000:0.0:1789760774.815622:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495904512:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760774.815669:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26288128, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd009 00010000:00010000:0.0:1789760774.815677:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454774a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.815693:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760774.815698:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.815704:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454774a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.815725:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454774a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760774.888400:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26288128 -> 26292224 00010000:00010000:2.0:1789760774.888421:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454774a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.888434:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454774a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.888444:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454774a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.888451:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94aa00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760774.888456:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454774a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.888465:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454774a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.888490:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454774a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.888498:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x493674835454774a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.888651:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973735 is committed 00000100:00100000:2.0:1789760774.888873:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495904512:12345-192.168.203.9@tcp:4:dd.0 Request processed in 73248us (73960us total) trans 4294973735 rc 0/0 00000100:00100000:2.0:1789760774.888884:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6447 00000100:00100000:2.0:1789760774.888902:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759102205626 time 759136326085 state 0x2 ref 0 00000100:00100000:0.0:1789760774.893168:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760774.894232:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495904640 00000100:00100000:1.0:1789760774.894245:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495904640 00000100:00100000:1.0:1789760774.894325:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759102205626 time 759136326085 state 0x0 ref 1 00000100:00100000:1.0:1789760774.894341:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759136586501 time 759136326085 state 0x1 ref 1 00000100:00100000:1.0:1789760774.894374:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759136586501 time 759215091468 state 0x0 ref 1 00000100:00100000:1.0:1789760774.894388:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495904640:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760774.894437:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26292224, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd009 00010000:00010000:1.0:1789760774.894446:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547751 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.894463:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760774.894468:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.894475:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547751 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.894489:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547751 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.929965:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973736 is committed 00002000:00010000:0.0:1789760774.930054:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26292224 -> 26296320 00010000:00010000:0.0:1789760774.930072:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547751 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.930085:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547751 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.930100:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547751 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.930106:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760774.930111:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547751 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.930119:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547751 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.930138:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547751 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.930146:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547751 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.930235:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495904640:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35846us (37071us total) trans 4294973736 rc 0/0 00000100:00100000:0.0:1789760774.930244:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6448 00000100:00100000:0.0:1789760774.930254:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759136586501 time 759215091468 state 0x2 ref 0 00000100:00100000:1.0:1789760774.938608:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760774.938722:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495904768 00000100:00100000:3.0:1789760774.938731:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495904768 00000100:00100000:3.0:1789760774.938807:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759136586501 time 759215091468 state 0x0 ref 1 00000100:00100000:3.0:1789760774.938823:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759215351884 time 759215091468 state 0x1 ref 1 00000100:00100000:3.0:1789760774.938893:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759215351884 time 759259609469 state 0x0 ref 1 00000100:00100000:3.0:1789760774.938907:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04929500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495904768:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760774.939461:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26296320, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd009 00010000:00010000:3.0:1789760774.939472:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547758 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.939488:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760774.939494:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.939501:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547758 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.939515:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547758 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.989255:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973737 is committed 00002000:00010000:0.0:1789760774.989419:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26296320 -> 26300416 00010000:00010000:0.0:1789760774.989436:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547758 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.989466:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547758 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.989475:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547758 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.989483:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760774.989488:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547758 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.989497:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547758 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.989517:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547758 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.989524:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547758 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.989637:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04929500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495904768:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50726us (51033us total) trans 4294973737 rc 0/0 00000100:00100000:0.0:1789760774.989647:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6449 00000100:00100000:0.0:1789760774.989658:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759215351884 time 759259609469 state 0x2 ref 0 00000100:00100000:3.0:1789760775.000259:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760775.000338:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495904896 00000100:00100000:0.0:1789760775.000349:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495904896 00000100:00100000:0.0:1789760775.000473:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759215351884 time 759259609469 state 0x0 ref 1 00000100:00100000:0.0:1789760775.000489:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759259869885 time 759259609469 state 0x1 ref 1 00000100:00100000:0.0:1789760775.000520:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759259869885 time 759321236567 state 0x0 ref 1 00000100:00100000:0.0:1789760775.000533:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495904896:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760775.000578:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26300416, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd009 00010000:00010000:0.0:1789760775.000587:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454775f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.000602:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760775.000607:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.000624:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454775f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.000638:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454775f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760775.001731:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789760775.006566:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760775.037594:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973738 is committed 00002000:00010000:0.0:1789760775.037702:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26300416 -> 26304512 00010000:00010000:0.0:1789760775.037717:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454775f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.037730:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454775f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.037746:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454775f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.037753:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760775.037759:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454775f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.037768:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454775f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.037866:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454775f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.037874:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454775f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.037969:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495904896:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37434us (37716us total) trans 4294973738 rc 0/0 00000100:00100000:0.0:1789760775.037980:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6450 00000100:00100000:0.0:1789760775.037989:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759259869885 time 759321236567 state 0x2 ref 0 00000100:00100000:2.0:1789760775.042199:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760775.043097:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495905024 00000100:00100000:0.0:1789760775.043108:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495905024 00000100:00100000:0.0:1789760775.043163:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759259869885 time 759321236567 state 0x0 ref 1 00000100:00100000:0.0:1789760775.043180:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759321496983 time 759321236567 state 0x1 ref 1 00000100:00100000:0.0:1789760775.043214:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759321496983 time 759363930458 state 0x0 ref 1 00000100:00100000:0.0:1789760775.043227:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495905024:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760775.043275:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26304512, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd009 00010000:00010000:0.0:1789760775.043298:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547766 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.043315:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760775.043319:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.043326:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547766 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.043340:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547766 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.077710:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973739 is committed 00002000:00010000:0.0:1789760775.077890:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26304512 -> 26308608 00010000:00010000:0.0:1789760775.077907:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547766 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.077920:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547766 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.077930:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547766 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.077937:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760775.077943:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547766 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.077952:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547766 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.077970:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547766 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.077978:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547766 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.078100:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495905024:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34870us (35904us total) trans 4294973739 rc 0/0 00000100:00100000:0.0:1789760775.078111:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6451 00000100:00100000:0.0:1789760775.078121:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759321496983 time 759363930458 state 0x2 ref 0 00000100:00100000:1.0:1789760775.084255:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760775.084600:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495905152 00000100:00100000:0.0:1789760775.084610:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495905152 00000100:00100000:0.0:1789760775.084669:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759321496983 time 759363930458 state 0x0 ref 1 00000100:00100000:0.0:1789760775.084683:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759364190874 time 759363930458 state 0x1 ref 1 00000100:00100000:0.0:1789760775.084716:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759364190874 time 759405432552 state 0x0 ref 1 00000100:00100000:0.0:1789760775.084728:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495905152:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760775.084777:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26308608, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd009 00010000:00010000:0.0:1789760775.084786:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454776d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.084816:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760775.084821:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.084828:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454776d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.084841:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454776d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.112093:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973740 is committed 00002000:00010000:1.0:1789760775.113960:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26308608 -> 26312704 00010000:00010000:1.0:1789760775.113977:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454776d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.113991:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454776d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.114001:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454776d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.114008:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760775.114013:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454776d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.114022:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454776d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.114043:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454776d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.114051:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x493674835454776d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.114243:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495905152:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29510us (29992us total) trans 4294973740 rc 0/0 00000100:00100000:1.0:1789760775.114254:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6452 00000100:00100000:1.0:1789760775.114264:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759364190874 time 759405432552 state 0x2 ref 0 00000100:00100000:0.0:1789760775.120543:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760775.120637:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495905280 00000100:00100000:1.0:1789760775.120647:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495905280 00000100:00100000:1.0:1789760775.120699:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759364190874 time 759405432552 state 0x0 ref 1 00000100:00100000:1.0:1789760775.120714:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759405692968 time 759405432552 state 0x1 ref 1 00000100:00100000:1.0:1789760775.120737:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759405692968 time 759441454660 state 0x0 ref 1 00000100:00100000:1.0:1789760775.120749:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495905280:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760775.120803:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26312704, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd009 00010000:00010000:1.0:1789760775.120811:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547774 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.120826:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760775.120830:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.120836:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547774 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.120850:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547774 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760775.152875:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26312704 -> 26316800 00000001:00080000:3.0:1789760775.152875:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973741 is committed 00010000:00010000:0.0:1789760775.152900:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547774 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.152913:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547774 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.152923:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547774 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.152930:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760775.152935:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547774 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.152944:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547774 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.152960:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547774 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.152966:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547774 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.153042:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495905280:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32290us (32504us total) trans 4294973741 rc 0/0 00000100:00100000:0.0:1789760775.153052:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6453 00000100:00100000:0.0:1789760775.153061:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759405692968 time 759441454660 state 0x2 ref 0 00000100:00100000:2.0:1789760775.162352:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760775.162632:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495905408 00000100:00100000:0.0:1789760775.162642:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495905408 00000100:00100000:0.0:1789760775.162698:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759405692968 time 759441454660 state 0x0 ref 1 00000100:00100000:0.0:1789760775.162714:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759441715076 time 759441454660 state 0x1 ref 1 00000100:00100000:0.0:1789760775.162754:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759441715076 time 759483470468 state 0x0 ref 1 00000100:00100000:0.0:1789760775.162766:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495905408:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760775.162833:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26316800, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd009 00010000:00010000:0.0:1789760775.162842:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454777b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.162857:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760775.162862:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.162869:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454777b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.162882:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454777b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.197303:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973742 is committed 00002000:00010000:0.0:1789760775.197371:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26316800 -> 26320896 00010000:00010000:0.0:1789760775.197387:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454777b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.197400:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454777b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.197410:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454777b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.197417:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760775.197563:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454777b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.197571:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454777b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.197588:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454777b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.197593:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454777b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.197674:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495905408:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34906us (35326us total) trans 4294973742 rc 0/0 00000100:00100000:0.0:1789760775.197685:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6454 00000100:00100000:0.0:1789760775.197694:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759441715076 time 759483470468 state 0x2 ref 0 00000100:00100000:1.0:1789760775.200683:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760775.200746:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495905536 00000100:00100000:3.0:1789760775.200756:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495905536 00000100:00100000:3.0:1789760775.200820:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759441715076 time 759483470468 state 0x0 ref 1 00000100:00100000:3.0:1789760775.200846:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759483730884 time 759483470468 state 0x1 ref 1 00000100:00100000:3.0:1789760775.200876:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759483730884 time 759521592998 state 0x0 ref 1 00000100:00100000:3.0:1789760775.200888:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495905536:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760775.200933:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26320896, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd009 00010000:00010000:3.0:1789760775.200942:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547782 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.200957:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760775.200962:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.200968:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547782 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.200982:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547782 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.246477:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973743 is committed 00002000:00010000:2.0:1789760775.246550:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26320896 -> 26324992 00010000:00010000:2.0:1789760775.246565:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547782 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.246578:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547782 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.246586:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547782 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.246594:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760775.246599:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547782 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.246609:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547782 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.246630:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547782 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.246637:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547782 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.246719:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495905536:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45828us (46040us total) trans 4294973743 rc 0/0 00000100:00100000:2.0:1789760775.246728:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6455 00000100:00100000:2.0:1789760775.246739:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759483730884 time 759521592998 state 0x2 ref 0 00000100:00100000:1.0:1789760775.251025:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760775.251132:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495905664 00000100:00100000:0.0:1789760775.251143:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495905664 00000100:00100000:0.0:1789760775.251274:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759483730884 time 759521592998 state 0x0 ref 1 00000100:00100000:0.0:1789760775.251477:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759521853414 time 759521592998 state 0x1 ref 1 00000100:00100000:0.0:1789760775.251518:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759521853414 time 759572235006 state 0x0 ref 1 00000100:00100000:0.0:1789760775.251531:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495905664:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760775.251587:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26324992, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd009 00010000:00010000:0.0:1789760775.251608:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547789 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.251631:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760775.251636:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.251642:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547789 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.251656:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547789 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.296243:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973744 is committed 00002000:00010000:1.0:1789760775.296381:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26324992 -> 26329088 00010000:00010000:1.0:1789760775.296396:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547789 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.296408:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547789 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.296416:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547789 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.296428:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760775.296433:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547789 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.296442:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547789 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.296461:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547789 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.296469:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547789 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.296602:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495905664:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45068us (45583us total) trans 4294973744 rc 0/0 00000100:00100000:1.0:1789760775.296612:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6456 00000100:00100000:1.0:1789760775.296622:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759521853414 time 759572235006 state 0x2 ref 0 00000100:00100000:2.0:1789760775.300856:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760775.300964:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495905792 00000100:00100000:0.0:1789760775.300974:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495905792 00000100:00100000:0.0:1789760775.301031:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759521853414 time 759572235006 state 0x0 ref 1 00000100:00100000:0.0:1789760775.301047:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759572495422 time 759572235006 state 0x1 ref 1 00000100:00100000:0.0:1789760775.301085:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759572495422 time 759621802008 state 0x0 ref 1 00000100:00100000:0.0:1789760775.301099:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495905792:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760775.301149:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26329088, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd009 00010000:00010000:0.0:1789760775.301158:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547790 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.301174:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760775.301178:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.301184:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547790 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.301197:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547790 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.338176:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973745 is committed 00002000:00010000:1.0:1789760775.338360:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26329088 -> 26333184 00010000:00010000:1.0:1789760775.338374:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547790 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.338386:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547790 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.338399:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547790 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.338406:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760775.338410:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547790 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.338418:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547790 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.338437:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547790 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.338445:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547790 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.338572:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495905792:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37471us (37718us total) trans 4294973745 rc 0/0 00000100:00100000:1.0:1789760775.338583:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6457 00000100:00100000:1.0:1789760775.338591:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759572495422 time 759621802008 state 0x2 ref 0 00000100:00100000:1.0:1789760775.343688:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760775.343801:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495905920 00000100:00100000:2.0:1789760775.343810:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495905920 00000100:00100000:2.0:1789760775.343863:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759572495422 time 759621802008 state 0x0 ref 1 00000100:00100000:2.0:1789760775.343879:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759622062424 time 759621802008 state 0x1 ref 1 00000100:00100000:2.0:1789760775.343949:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759622062424 time 759664665365 state 0x0 ref 1 00000100:00100000:2.0:1789760775.343962:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495905920:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760775.344010:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26333184, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd009 00010000:00010000:2.0:1789760775.344019:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547797 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.344036:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:2.0:1789760775.344040:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.344046:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547797 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.344061:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547797 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760775.367016:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26333184 -> 26337280 00010000:00010000:2.0:1789760775.367035:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547797 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.367049:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547797 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.367059:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547797 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.367066:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760775.367071:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547797 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.367080:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547797 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.367103:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547797 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.367110:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547797 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.367200:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495905920:12345-192.168.203.9@tcp:4:dd.0 Request processed in 23235us (23516us total) trans 4294973746 rc 0/0 00000100:00100000:2.0:1789760775.367211:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6458 00000100:00100000:2.0:1789760775.367221:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759622062424 time 759664665365 state 0x2 ref 0 00000001:00080000:0.0:1789760775.367303:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973746 is committed 00000100:00100000:2.0:1789760775.369957:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760775.370022:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495906048 00000100:00100000:1.0:1789760775.370032:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495906048 00000100:00100000:1.0:1789760775.370098:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759622062424 time 759664665365 state 0x0 ref 1 00000100:00100000:1.0:1789760775.370118:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759664925781 time 759664665365 state 0x1 ref 1 00000100:00100000:1.0:1789760775.370143:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759664925781 time 759690860099 state 0x0 ref 1 00000100:00100000:1.0:1789760775.370156:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495906048:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760775.370204:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26337280, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd009 00010000:00010000:1.0:1789760775.370213:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454779e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.370229:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760775.370234:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.370241:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454779e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.370255:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454779e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760775.394626:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26337280 -> 26341376 00010000:00010000:0.0:1789760775.394646:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454779e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.394676:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454779e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.394684:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454779e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.394692:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760775.394697:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454779e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.394705:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454779e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.394723:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454779e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.394730:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454779e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.394844:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495906048:12345-192.168.203.9@tcp:4:dd.0 Request processed in 24686us (24891us total) trans 4294973747 rc 0/0 00000100:00100000:0.0:1789760775.394854:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6459 00000100:00100000:0.0:1789760775.394864:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759664925781 time 759690860099 state 0x2 ref 0 00000001:00080000:3.0:1789760775.395453:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973747 is committed 00000100:00100000:2.0:1789760775.402559:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760775.402808:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495906176 00000100:00100000:1.0:1789760775.402818:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495906176 00000100:00100000:1.0:1789760775.402989:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759664925781 time 759690860099 state 0x0 ref 1 00000100:00100000:1.0:1789760775.403007:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759691120515 time 759690860099 state 0x1 ref 1 00000100:00100000:1.0:1789760775.403040:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759691120515 time 759723756857 state 0x0 ref 1 00000100:00100000:1.0:1789760775.403055:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02768e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495906176:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760775.403195:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26341376, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd009 00010000:00010000:1.0:1789760775.403205:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545477a5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.403220:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760775.403225:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.403231:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545477a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.403245:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545477a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.437628:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973748 is committed 00002000:00010000:3.0:1789760775.437880:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26341376 -> 26345472 00010000:00010000:3.0:1789760775.437898:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545477a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.437911:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545477a5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.437927:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545477a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.437935:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760775.437940:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545477a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.437950:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545477a5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.437971:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545477a5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.437978:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545477a5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.438078:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02768e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495906176:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35023us (35522us total) trans 4294973748 rc 0/0 00000100:00100000:3.0:1789760775.438089:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6460 00000100:00100000:3.0:1789760775.438098:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759691120515 time 759723756857 state 0x2 ref 0 00000100:00100000:2.0:1789760775.445968:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760775.446081:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495906304 00000100:00100000:0.0:1789760775.446093:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495906304 00000100:00100000:0.0:1789760775.446150:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759691120515 time 759723756857 state 0x0 ref 1 00000100:00100000:0.0:1789760775.446166:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759724017273 time 759723756857 state 0x1 ref 1 00000100:00100000:0.0:1789760775.446194:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759724017273 time 759766911153 state 0x0 ref 1 00000100:00100000:0.0:1789760775.446208:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495906304:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760775.446257:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26345472, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd009 00010000:00010000:0.0:1789760775.446266:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545477ac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.446365:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760775.446370:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.446377:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545477ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.446391:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545477ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760775.497396:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26345472 -> 26349568 00010000:00010000:1.0:1789760775.497419:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545477ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.497434:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545477ac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.497517:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545477ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.497525:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760775.497530:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545477ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.497540:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545477ac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.497561:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545477ac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.497570:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545477ac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.497742:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973749 is committed 00000100:00100000:1.0:1789760775.497915:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495906304:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51705us (51951us total) trans 4294973749 rc 0/0 00000100:00100000:1.0:1789760775.497929:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6461 00000100:00100000:1.0:1789760775.497940:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759724017273 time 759766911153 state 0x2 ref 0 00000100:00100000:2.0:1789760775.506098:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760775.506239:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495906432 00000100:00100000:0.0:1789760775.506249:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495906432 00000100:00100000:0.0:1789760775.506553:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759724017273 time 759766911153 state 0x0 ref 1 00000100:00100000:0.0:1789760775.506568:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759767171569 time 759766911153 state 0x1 ref 1 00000100:00100000:1.0:1789760775.506706:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759767171569 time 759827420338 state 0x0 ref 1 00000100:00100000:1.0:1789760775.506722:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495906432:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760775.506773:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26349568, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd009 00010000:00010000:1.0:1789760775.506782:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545477b3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.506810:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760775.506815:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.506821:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545477b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.506836:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545477b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760775.532357:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26349568 -> 26353664 00010000:00010000:3.0:1789760775.532378:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545477b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.532397:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545477b3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.532407:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545477b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.532415:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760775.532420:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545477b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.532434:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545477b3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.532451:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973750 is committed 00010000:00010000:3.0:1789760775.532524:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545477b3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.532533:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x49367483545477b3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.532623:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495906432:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25899us (26528us total) trans 4294973750 rc 0/0 00000100:00100000:3.0:1789760775.532634:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6462 00000100:00100000:3.0:1789760775.532644:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759767171569 time 759827420338 state 0x2 ref 0 00000100:00100000:1.0:1789760775.538485:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760775.538644:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495906560 00000100:00100000:3.0:1789760775.538655:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495906560 00000100:00100000:3.0:1789760775.538709:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759767171569 time 759827420338 state 0x0 ref 1 00000100:00100000:3.0:1789760775.538724:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759827680754 time 759827420338 state 0x1 ref 1 00000100:00100000:3.0:1789760775.538748:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759827680754 time 759859464733 state 0x0 ref 1 00000100:00100000:3.0:1789760775.538760:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495906560:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760775.538819:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26353664, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd009 00010000:00010000:3.0:1789760775.538829:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477ba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.538855:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760775.538860:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.539047:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.539064:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760775.570203:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26353664 -> 26357760 00010000:00010000:3.0:1789760775.570223:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.570238:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477ba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.570253:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.570261:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760775.570266:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.570276:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477ba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.570328:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477ba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.570336:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477ba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.570421:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495906560:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31659us (31952us total) trans 4294973751 rc 0/0 00000100:00100000:3.0:1789760775.570432:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6463 00000100:00100000:3.0:1789760775.570442:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759827680754 time 759859464733 state 0x2 ref 0 00000001:00080000:0.0:1789760775.571023:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973751 is committed 00000100:00100000:2.0:1789760775.575209:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760775.575318:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495906688 00000100:00100000:1.0:1789760775.575327:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495906688 00000100:00100000:1.0:1789760775.575382:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759827680754 time 759859464733 state 0x0 ref 1 00000100:00100000:1.0:1789760775.575396:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759859725149 time 759859464733 state 0x1 ref 1 00000100:00100000:1.0:1789760775.575423:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759859725149 time 759896139811 state 0x0 ref 1 00000100:00100000:1.0:1789760775.575433:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495906688:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760775.575477:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26357760, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd009 00010000:00010000:1.0:1789760775.575486:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545477c1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.575502:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760775.575506:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.575512:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545477c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.575525:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545477c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.620075:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973752 is committed 00002000:00010000:1.0:1789760775.620255:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26357760 -> 26361856 00010000:00010000:1.0:1789760775.620270:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545477c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.620301:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545477c1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.620310:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545477c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.620317:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760775.620322:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545477c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.620330:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545477c1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.620421:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545477c1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.620430:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545477c1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.620560:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495906688:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45122us (45355us total) trans 4294973752 rc 0/0 00000100:00100000:1.0:1789760775.620570:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6464 00000100:00100000:1.0:1789760775.620581:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759859725149 time 759896139811 state 0x2 ref 0 00000100:00100000:1.0:1789760775.624418:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760775.624510:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495906816 00000100:00100000:3.0:1789760775.624520:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495906816 00000100:00100000:3.0:1789760775.624577:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759859725149 time 759896139811 state 0x0 ref 1 00000100:00100000:3.0:1789760775.624592:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759896400227 time 759896139811 state 0x1 ref 1 00000100:00100000:3.0:1789760775.624672:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759896400227 time 759945387219 state 0x0 ref 1 00000100:00100000:3.0:1789760775.624685:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495906816:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760775.624733:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26361856, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd009 00010000:00010000:3.0:1789760775.624742:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477c8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.624759:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760775.624773:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.624781:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.624804:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.652677:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973753 is committed 00002000:00010000:3.0:1789760775.652761:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26361856 -> 26365952 00002000:00010000:3.0:1789760775.652773:0:13059:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760774 -> 1789760775 00002000:00010000:3.0:1789760775.652776:0:13059:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760774 -> 1789760775 00010000:00010000:3.0:1789760775.652798:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.652811:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477c8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.652820:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.652828:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760775.652833:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.652842:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.652861:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477c8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.652868:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477c8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.652993:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495906816:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28305us (28579us total) trans 4294973753 rc 0/0 00000100:00100000:3.0:1789760775.653004:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6465 00000100:00100000:3.0:1789760775.653013:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759896400227 time 759945387219 state 0x2 ref 0 00000100:00100000:2.0:1789760775.655725:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760775.656039:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495906944 00000100:00100000:3.0:1789760775.656050:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495906944 00000100:00100000:3.0:1789760775.656105:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759896400227 time 759945387219 state 0x0 ref 1 00000100:00100000:3.0:1789760775.656419:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759945647635 time 759945387219 state 0x1 ref 1 00000100:00100000:2.0:1789760775.656511:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759945647635 time 759977225095 state 0x0 ref 1 00000100:00100000:2.0:1789760775.656526:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036399c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495906944:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760775.656575:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26365952, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd009 00010000:00010000:2.0:1789760775.656585:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545477cf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.656601:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:2.0:1789760775.656606:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.656612:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545477cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.656623:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545477cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.689355:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973754 is committed 00002000:00010000:3.0:1789760775.689581:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26365952 -> 26370048 00010000:00010000:3.0:1789760775.689597:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545477cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.689610:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545477cf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.689619:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545477cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.689627:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760775.689632:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545477cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.689640:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545477cf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.689661:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545477cf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.689669:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545477cf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.689817:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036399c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495906944:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33290us (34096us total) trans 4294973754 rc 0/0 00000100:00100000:3.0:1789760775.689829:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6466 00000100:00100000:3.0:1789760775.689838:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759945647635 time 759977225095 state 0x2 ref 0 00000100:00100000:2.0:1789760775.697606:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760775.697715:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495907072 00000100:00100000:1.0:1789760775.697724:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495907072 00000100:00100000:1.0:1789760775.697795:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759945647635 time 759977225095 state 0x0 ref 1 00000100:00100000:1.0:1789760775.697812:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759977485511 time 759977225095 state 0x1 ref 1 00000100:00100000:1.0:1789760775.697845:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759977485511 time 760018561715 state 0x0 ref 1 00000100:00100000:1.0:1789760775.697858:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495907072:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760775.697911:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26370048, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd009 00010000:00010000:1.0:1789760775.697921:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545477d6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.697938:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760775.697943:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.697950:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545477d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.697963:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545477d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.723033:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973755 is committed 00002000:00010000:0.0:1789760775.723132:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26370048 -> 26374144 00010000:00010000:0.0:1789760775.723148:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545477d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.723161:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545477d6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.723171:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545477d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.723178:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760775.723183:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545477d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.723192:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545477d6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.723212:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545477d6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.723220:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545477d6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.723350:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495907072:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25490us (25749us total) trans 4294973755 rc 0/0 00000100:00100000:0.0:1789760775.723360:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6467 00000100:00100000:0.0:1789760775.723370:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759977485511 time 760018561715 state 0x2 ref 0 00000100:00100000:1.0:1789760775.727697:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760775.727824:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495907200 00000100:00100000:3.0:1789760775.727834:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495907200 00000100:00100000:3.0:1789760775.727889:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 759977485511 time 760018561715 state 0x0 ref 1 00000100:00100000:3.0:1789760775.727904:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760018822131 time 760018561715 state 0x1 ref 1 00000100:00100000:3.0:1789760775.727935:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760018822131 time 760048651492 state 0x0 ref 1 00000100:00100000:3.0:1789760775.727947:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495907200:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760775.727993:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26374144, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd009 00010000:00010000:3.0:1789760775.728002:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477dd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.728016:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760775.728020:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.728027:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.728040:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.767612:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973756 is committed 00002000:00010000:3.0:1789760775.767711:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26374144 -> 26378240 00010000:00010000:3.0:1789760775.767727:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.767741:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477dd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.767755:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.767775:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760775.767780:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.767798:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477dd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.767818:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477dd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.767826:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477dd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.767917:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495907200:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39966us (40223us total) trans 4294973756 rc 0/0 00000100:00100000:3.0:1789760775.767929:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6468 00000100:00100000:3.0:1789760775.767938:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760018822131 time 760048651492 state 0x2 ref 0 00000100:00100000:1.0:1789760775.771062:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760775.772859:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495907328 00000100:00100000:3.0:1789760775.772871:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495907328 00000100:00100000:3.0:1789760775.772995:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760018822131 time 760048651492 state 0x0 ref 1 00000100:00100000:3.0:1789760775.773008:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760048911908 time 760048651492 state 0x1 ref 1 00000100:00100000:3.0:1789760775.773035:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760048911908 time 760093753479 state 0x0 ref 1 00000100:00100000:3.0:1789760775.773046:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495907328:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760775.773198:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26378240, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd009 00010000:00010000:3.0:1789760775.773208:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545477e4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.773228:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760775.773233:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.773240:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545477e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.773254:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545477e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760775.798912:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26378240 -> 26382336 00010000:00010000:1.0:1789760775.798937:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545477e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.798953:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545477e4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.798962:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545477e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.798970:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760775.798975:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545477e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.798985:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545477e4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.799008:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545477e4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.799016:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545477e4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.799124:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495907328:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26073us (28065us total) trans 4294973757 rc 0/0 00000100:00100000:1.0:1789760775.799136:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6469 00000100:00100000:1.0:1789760775.799145:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760048911908 time 760093753479 state 0x2 ref 0 00000001:00080000:0.0:1789760775.801228:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973757 is committed 00000100:00100000:1.0:1789760775.802490:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760775.802641:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495907456 00000100:00100000:0.0:1789760775.802651:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495907456 00000100:00100000:0.0:1789760775.802706:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760048911908 time 760093753479 state 0x0 ref 1 00000100:00100000:0.0:1789760775.802722:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760094013895 time 760093753479 state 0x1 ref 1 00000100:00100000:0.0:1789760775.802758:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760094013895 time 760123474993 state 0x0 ref 1 00000100:00100000:0.0:1789760775.802843:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495907456:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760775.802887:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26382336, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd009 00010000:00010000:0.0:1789760775.802896:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545477eb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.802926:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760775.802931:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.802937:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545477eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.802951:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545477eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.832834:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973758 is committed 00002000:00010000:0.0:1789760775.832938:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26382336 -> 26386432 00010000:00010000:0.0:1789760775.832954:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545477eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.832968:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545477eb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.832977:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545477eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.832984:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760775.832989:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545477eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.832998:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545477eb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.833018:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545477eb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.833025:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545477eb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.833192:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495907456:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30347us (30703us total) trans 4294973758 rc 0/0 00000100:00100000:0.0:1789760775.833203:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6470 00000100:00100000:0.0:1789760775.833213:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760094013895 time 760123474993 state 0x2 ref 0 00000100:00100000:2.0:1789760775.837026:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760775.837081:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495907584 00000100:00100000:3.0:1789760775.837091:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495907584 00000100:00100000:3.0:1789760775.837203:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760094013895 time 760123474993 state 0x0 ref 1 00000100:00100000:3.0:1789760775.837218:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760123735409 time 760123474993 state 0x1 ref 1 00000100:00100000:3.0:1789760775.837250:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760123735409 time 760157967256 state 0x0 ref 1 00000100:00100000:3.0:1789760775.837262:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495907584:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760775.837336:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26386432, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd009 00010000:00010000:3.0:1789760775.837347:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477f2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.837362:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760775.837367:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.837374:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.837396:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.894927:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973759 is committed 00002000:00010000:1.0:1789760775.895031:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26386432 -> 26390528 00010000:00010000:1.0:1789760775.895047:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.895060:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477f2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.895069:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.895076:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760775.895080:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.895089:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477f2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.895182:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477f2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.895191:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545477f2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.895314:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495907584:12345-192.168.203.9@tcp:4:dd.0 Request processed in 58017us (58262us total) trans 4294973759 rc 0/0 00000100:00100000:1.0:1789760775.895325:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6471 00000100:00100000:1.0:1789760775.895337:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760123735409 time 760157967256 state 0x2 ref 0 00000100:00100000:1.0:1789760775.899481:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760775.899568:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495907712 00000100:00100000:3.0:1789760775.899577:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495907712 00000100:00100000:3.0:1789760775.899627:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760123735409 time 760157967256 state 0x0 ref 1 00000100:00100000:3.0:1789760775.899655:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760158227672 time 760157967256 state 0x1 ref 1 00000100:00100000:3.0:1789760775.899683:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760158227672 time 760220399547 state 0x0 ref 1 00000100:00100000:3.0:1789760775.899702:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034ce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495907712:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760775.899747:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26390528, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd009 00010000:00010000:3.0:1789760775.899755:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545477f9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.899770:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760775.899774:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.899779:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545477f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.899805:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545477f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.928315:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973760 is committed 00002000:00010000:1.0:1789760775.928430:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26390528 -> 26394624 00010000:00010000:1.0:1789760775.928448:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545477f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.928462:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545477f9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.928471:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545477f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.928541:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760775.928547:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545477f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.928555:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545477f9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.928574:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545477f9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.928581:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545477f9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.928720:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034ce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495907712:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29014us (29241us total) trans 4294973760 rc 0/0 00000100:00100000:1.0:1789760775.928734:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6472 00000100:00100000:1.0:1789760775.928745:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760158227672 time 760220399547 state 0x2 ref 0 00000100:00100000:2.0:1789760775.934493:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760775.934604:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495907840 00000100:00100000:0.0:1789760775.934614:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495907840 00000100:00100000:0.0:1789760775.934672:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760158227672 time 760220399547 state 0x0 ref 1 00000100:00100000:0.0:1789760775.934690:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760220659963 time 760220399547 state 0x1 ref 1 00000100:00100000:0.0:1789760775.934719:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760220659963 time 760255435276 state 0x0 ref 1 00000100:00100000:0.0:1789760775.934732:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495907840:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760775.934779:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26394624, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd009 00010000:00010000:0.0:1789760775.934799:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547800 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.934816:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760775.934820:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.934826:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547800 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.934842:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547800 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.957083:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973761 is committed 00002000:00010000:2.0:1789760775.957319:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26394624 -> 26398720 00010000:00010000:2.0:1789760775.957334:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547800 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.957346:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547800 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.957355:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547800 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.957367:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760775.957371:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547800 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.957379:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547800 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.957397:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547800 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.957404:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547800 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.957570:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495907840:12345-192.168.203.9@tcp:4:dd.0 Request processed in 22835us (23083us total) trans 4294973761 rc 0/0 00000100:00100000:2.0:1789760775.957581:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6473 00000100:00100000:2.0:1789760775.957590:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760220659963 time 760255435276 state 0x2 ref 0 00000100:00100000:2.0:1789760775.965652:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760775.965793:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495907968 00000100:00100000:1.0:1789760775.965803:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495907968 00000100:00100000:1.0:1789760775.965921:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760220659963 time 760255435276 state 0x0 ref 1 00000100:00100000:1.0:1789760775.965936:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760255695692 time 760255435276 state 0x1 ref 1 00000100:00100000:1.0:1789760775.966000:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760255695692 time 760286716280 state 0x0 ref 1 00000100:00100000:1.0:1789760775.966012:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495907968:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760775.966061:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26398720, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd009 00010000:00010000:1.0:1789760775.966069:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547807 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.966085:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760775.966089:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.966096:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547807 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.966110:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547807 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.013792:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973762 is committed 00002000:00010000:3.0:1789760776.013999:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26398720 -> 26402816 00010000:00010000:3.0:1789760776.014014:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547807 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.014026:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547807 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.014035:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547807 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.014043:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760776.014048:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547807 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.014058:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547807 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.014077:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547807 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.014084:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547807 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.014415:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495907968:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48399us (48768us total) trans 4294973762 rc 0/0 00000100:00100000:3.0:1789760776.014427:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6474 00000100:00100000:3.0:1789760776.014437:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760255695692 time 760286716280 state 0x2 ref 0 00000100:00100000:1.0:1789760776.018905:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760776.019039:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495908096 00000100:00100000:3.0:1789760776.019048:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495908096 00000100:00100000:3.0:1789760776.019114:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760255695692 time 760286716280 state 0x0 ref 1 00000100:00100000:3.0:1789760776.019130:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760286976696 time 760286716280 state 0x1 ref 1 00000100:00100000:3.0:1789760776.019162:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760286976696 time 760339878907 state 0x0 ref 1 00000100:00100000:3.0:1789760776.019175:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495908096:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760776.019225:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26402816, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd009 00010000:00010000:3.0:1789760776.019234:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454780e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.019252:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760776.019257:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.019275:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454780e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.019331:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454780e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.030449:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760776.067429:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973763 is committed 00002000:00010000:0.0:1789760776.067524:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26402816 -> 26406912 00010000:00010000:0.0:1789760776.067542:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454780e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.067556:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454780e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.067573:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454780e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.067580:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760776.067586:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454780e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.067596:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454780e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.067617:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454780e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.067641:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454780e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.067736:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495908096:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48558us (48836us total) trans 4294973763 rc 0/0 00000100:00100000:0.0:1789760776.067746:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6475 00000100:00100000:0.0:1789760776.067757:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760286976696 time 760339878907 state 0x2 ref 0 00000100:00100000:2.0:1789760776.073676:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760776.073739:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495908224 00000100:00100000:0.0:1789760776.073749:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495908224 00000100:00100000:0.0:1789760776.073813:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760286976696 time 760339878907 state 0x0 ref 1 00000100:00100000:0.0:1789760776.073830:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760340139323 time 760339878907 state 0x1 ref 1 00000100:00100000:0.0:1789760776.073864:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760340139323 time 760394580864 state 0x0 ref 1 00000100:00100000:0.0:1789760776.073877:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495908224:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760776.073925:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26406912, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd009 00010000:00010000:0.0:1789760776.073934:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547815 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.073949:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760776.073954:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.073961:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547815 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.073985:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547815 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.117512:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973764 is committed 00002000:00010000:1.0:1789760776.117624:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26406912 -> 26411008 00010000:00010000:1.0:1789760776.117640:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547815 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.117652:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547815 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.117661:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547815 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.117667:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760776.117671:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547815 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.117679:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547815 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.117698:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547815 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.117705:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547815 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.117797:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495908224:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43919us (44127us total) trans 4294973764 rc 0/0 00000100:00100000:1.0:1789760776.120625:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6476 00000100:00100000:1.0:1789760776.120639:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760340139323 time 760394580864 state 0x2 ref 0 00000100:00100000:2.0:1789760776.121734:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760776.122147:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495908352 00000100:00100000:3.0:1789760776.122157:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495908352 00000100:00100000:3.0:1789760776.122211:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760340139323 time 760394580864 state 0x0 ref 1 00000100:00100000:3.0:1789760776.122226:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760394841280 time 760394580864 state 0x1 ref 1 00000100:00100000:3.0:1789760776.122256:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760394841280 time 760442972820 state 0x0 ref 1 00000100:00100000:3.0:1789760776.122268:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04575500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495908352:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760776.122536:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26411008, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd009 00010000:00010000:3.0:1789760776.122546:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454781c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.122562:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760776.122567:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.122574:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454781c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.122663:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454781c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.163686:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973765 is committed 00002000:00010000:1.0:1789760776.163722:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26411008 -> 26415104 00010000:00010000:1.0:1789760776.163741:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454781c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.163755:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454781c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.163765:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454781c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.163772:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760776.163777:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454781c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.163799:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454781c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.163818:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454781c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.163826:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454781c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.163965:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04575500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495908352:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41693us (42230us total) trans 4294973765 rc 0/0 00000100:00100000:1.0:1789760776.163976:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6477 00000100:00100000:1.0:1789760776.163986:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760394841280 time 760442972820 state 0x2 ref 0 00000100:00100000:2.0:1789760776.170401:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760776.170490:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495908480 00000100:00100000:0.0:1789760776.170500:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495908480 00000100:00100000:0.0:1789760776.170554:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760394841280 time 760442972820 state 0x0 ref 1 00000100:00100000:0.0:1789760776.170569:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760443233236 time 760442972820 state 0x1 ref 1 00000100:00100000:0.0:1789760776.170633:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760443233236 time 760491349377 state 0x0 ref 1 00000100:00100000:0.0:1789760776.170645:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04577b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495908480:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760776.170690:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26415104, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd009 00010000:00010000:0.0:1789760776.170700:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547823 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.170714:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760776.170719:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.170724:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547823 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.170737:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547823 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.225606:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973766 is committed 00002000:00010000:0.0:1789760776.225751:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26415104 -> 26419200 00010000:00010000:0.0:1789760776.225767:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547823 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.225790:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547823 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.225801:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547823 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.225809:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760776.225813:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547823 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.225822:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547823 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.225847:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547823 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.225855:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547823 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.226018:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04577b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495908480:12345-192.168.203.9@tcp:4:dd.0 Request processed in 55369us (55680us total) trans 4294973766 rc 0/0 00000100:00100000:0.0:1789760776.226028:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6478 00000100:00100000:0.0:1789760776.226037:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760443233236 time 760491349377 state 0x2 ref 0 00000100:00100000:2.0:1789760776.231933:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760776.232044:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495908608 00000100:00100000:3.0:1789760776.232054:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495908608 00000100:00100000:3.0:1789760776.232112:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760443233236 time 760491349377 state 0x0 ref 1 00000100:00100000:3.0:1789760776.232128:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760491609793 time 760491349377 state 0x1 ref 1 00000100:00100000:3.0:1789760776.232164:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760491609793 time 760552880110 state 0x0 ref 1 00000100:00100000:3.0:1789760776.232177:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01c8d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495908608:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760776.232225:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26419200, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd009 00010000:00010000:3.0:1789760776.232234:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454782a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.232250:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760776.232255:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.232262:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454782a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.232277:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454782a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760776.274352:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26419200 -> 26423296 00010000:00010000:0.0:1789760776.274373:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454782a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.274386:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454782a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.274395:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454782a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.274404:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760776.274408:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454782a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.274422:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454782a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.274441:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454782a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.274449:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454782a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.274532:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01c8d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495908608:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42352us (42604us total) trans 4294973767 rc 0/0 00000100:00100000:0.0:1789760776.274543:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6479 00000100:00100000:0.0:1789760776.274553:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760491609793 time 760552880110 state 0x2 ref 0 00000001:00080000:1.0:1789760776.274638:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973767 is committed 00000100:00100000:2.0:1789760776.281586:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760776.281689:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495908736 00000100:00100000:1.0:1789760776.281698:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495908736 00000100:00100000:1.0:1789760776.281755:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760491609793 time 760552880110 state 0x0 ref 1 00000100:00100000:1.0:1789760776.281789:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760553140526 time 760552880110 state 0x1 ref 1 00000100:00100000:1.0:1789760776.281816:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760553140526 time 760602532730 state 0x0 ref 1 00000100:00100000:1.0:1789760776.281828:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495908736:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760776.281875:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26423296, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd009 00010000:00010000:1.0:1789760776.281884:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547831 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.281901:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760776.281906:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.281912:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547831 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.281925:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547831 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.320186:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973768 is committed 00002000:00010000:1.0:1789760776.320347:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26423296 -> 26427392 00010000:00010000:1.0:1789760776.320363:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547831 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.320377:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547831 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.320386:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547831 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.320394:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760776.320399:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547831 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.320408:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547831 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.320427:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547831 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.320435:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547831 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.320519:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495908736:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38688us (38937us total) trans 4294973768 rc 0/0 00000100:00100000:1.0:1789760776.320531:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6480 00000100:00100000:1.0:1789760776.320542:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760553140526 time 760602532730 state 0x2 ref 0 00000100:00100000:2.0:1789760776.327494:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760776.327590:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495908864 00000100:00100000:0.0:1789760776.327599:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495908864 00000100:00100000:0.0:1789760776.327655:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760553140526 time 760602532730 state 0x0 ref 1 00000100:00100000:0.0:1789760776.327670:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760602793146 time 760602532730 state 0x1 ref 1 00000100:00100000:0.0:1789760776.327704:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760602793146 time 760648420799 state 0x0 ref 1 00000100:00100000:0.0:1789760776.327716:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495908864:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760776.327774:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26427392, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd009 00010000:00010000:0.0:1789760776.327793:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547838 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.327809:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760776.327814:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.327819:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547838 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.327832:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547838 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.377682:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973769 is committed 00002000:00010000:2.0:1789760776.377874:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26427392 -> 26431488 00010000:00010000:2.0:1789760776.377890:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547838 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.377902:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547838 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.377911:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547838 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.377918:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760776.377922:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547838 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.377929:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547838 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.377947:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547838 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.377953:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547838 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.378042:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495908864:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50323us (50551us total) trans 4294973769 rc 0/0 00000100:00100000:2.0:1789760776.378053:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6481 00000100:00100000:2.0:1789760776.378062:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760602793146 time 760648420799 state 0x2 ref 0 00000100:00100000:3.0:1789760776.384147:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760776.384473:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495908992 00000100:00100000:0.0:1789760776.384484:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495908992 00000100:00100000:1.0:1789760776.384684:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760602793146 time 760648420799 state 0x0 ref 1 00000100:00100000:1.0:1789760776.384700:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760648681215 time 760648420799 state 0x1 ref 1 00000100:00100000:1.0:1789760776.384732:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760648681215 time 760705449093 state 0x0 ref 1 00000100:00100000:1.0:1789760776.384746:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495908992:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760776.385411:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26431488, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd009 00010000:00010000:1.0:1789760776.385422:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454783f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.385437:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760776.385442:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.385450:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454783f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.385463:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454783f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.418739:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973770 is committed 00002000:00010000:3.0:1789760776.418906:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26431488 -> 26435584 00010000:00010000:3.0:1789760776.418921:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454783f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.419066:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454783f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.419078:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454783f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.419088:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760776.419095:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454783f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.419105:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454783f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.419128:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454783f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.419137:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454783f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.419379:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495908992:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34631us (35235us total) trans 4294973770 rc 0/0 00000100:00100000:3.0:1789760776.419390:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6482 00000100:00100000:3.0:1789760776.419400:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760648681215 time 760705449093 state 0x2 ref 0 00000100:00100000:1.0:1789760776.422438:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760776.422544:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495909120 00000100:00100000:2.0:1789760776.422552:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495909120 00000100:00100000:2.0:1789760776.422600:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760648681215 time 760705449093 state 0x0 ref 1 00000100:00100000:2.0:1789760776.422615:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760705709509 time 760705449093 state 0x1 ref 1 00000100:00100000:2.0:1789760776.422669:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760705709509 time 760743385489 state 0x0 ref 1 00000100:00100000:2.0:1789760776.422681:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495909120:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760776.422725:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26435584, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd009 00010000:00010000:2.0:1789760776.422734:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547846 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.422750:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760776.422755:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.422829:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547846 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.422846:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547846 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.448138:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973771 is committed 00002000:00010000:2.0:1789760776.448232:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26435584 -> 26439680 00010000:00010000:2.0:1789760776.448248:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547846 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.448262:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547846 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.448271:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547846 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.448302:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760776.448308:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547846 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.448317:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547846 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.448335:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547846 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.448350:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547846 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.448475:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495909120:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25790us (26041us total) trans 4294973771 rc 0/0 00000100:00100000:2.0:1789760776.448486:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6483 00000100:00100000:2.0:1789760776.448498:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760705709509 time 760743385489 state 0x2 ref 0 00000100:00100000:2.0:1789760776.454618:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760776.454892:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495909248 00000100:00100000:0.0:1789760776.454902:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495909248 00000100:00100000:0.0:1789760776.454962:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760705709509 time 760743385489 state 0x0 ref 1 00000100:00100000:0.0:1789760776.454978:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760743645905 time 760743385489 state 0x1 ref 1 00000100:00100000:0.0:1789760776.455057:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760743645905 time 760775772483 state 0x0 ref 1 00000100:00100000:0.0:1789760776.455070:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495909248:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760776.455123:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26439680, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd009 00010000:00010000:0.0:1789760776.455133:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454784d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.455148:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760776.455152:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.455158:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454784d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.455171:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454784d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.478858:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789760776.478880:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f0276fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520314112:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760776.478961:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760776.479239:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520314112 00000100:00100000:1.0:1789760776.479247:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520314112 00000100:00100000:1.0:1789760776.479319:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 213 00000100:00100000:1.0:1789760776.479328:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520314112:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760776.479351:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527761408 pending 1703936 free 3831496704 avail 3787456512 00000100:00100000:1.0:1789760776.479399:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520314112:12345-0@lo:13:osp-pre-0-0.0 Request processed in 72us (437us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760776.479409:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 213 00000100:00100000:1.0:1789760776.479454:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f0276fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520314112:0@lo:13:osp-pre-0-0.0 00000100:00080000:2.0:1789760776.479672:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760776.486145:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973772 is committed 00002000:00010000:0.0:1789760776.486412:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26439680 -> 26443776 00010000:00010000:0.0:1789760776.486427:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454784d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.486440:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454784d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.486449:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454784d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.486457:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760776.486462:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454784d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.486469:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454784d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.486485:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454784d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.486492:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454784d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.486786:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495909248:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31706us (32165us total) trans 4294973772 rc 0/0 00000100:00100000:0.0:1789760776.486798:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6484 00000100:00100000:0.0:1789760776.486809:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760743645905 time 760775772483 state 0x2 ref 0 00000100:00100000:3.0:1789760776.491751:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760776.493159:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495909376 00000100:00100000:1.0:1789760776.493173:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495909376 00000100:00100000:1.0:1789760776.493230:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760743645905 time 760775772483 state 0x0 ref 1 00000100:00100000:1.0:1789760776.493243:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760776032899 time 760775772483 state 0x1 ref 1 00000100:00100000:1.0:1789760776.493276:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760776032899 time 760813992308 state 0x0 ref 1 00000100:00100000:1.0:1789760776.493304:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495909376:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760776.493349:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26443776, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd009 00010000:00010000:1.0:1789760776.493358:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354547854 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.493374:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760776.493378:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.493384:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354547854 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.493398:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354547854 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.531068:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973773 is committed 00002000:00010000:3.0:1789760776.531228:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26443776 -> 26447872 00010000:00010000:3.0:1789760776.531244:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354547854 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.531256:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354547854 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.531269:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354547854 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.531276:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760776.531309:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354547854 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.531318:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354547854 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.531336:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354547854 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.531344:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354547854 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.531430:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495909376:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38126us (39684us total) trans 4294973773 rc 0/0 00000100:00100000:3.0:1789760776.531440:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6485 00000100:00100000:3.0:1789760776.531450:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760776032899 time 760813992308 state 0x2 ref 0 00000100:00100000:2.0:1789760776.534155:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760776.534254:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495909504 00000100:00100000:1.0:1789760776.534263:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495909504 00000100:00100000:1.0:1789760776.534344:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760776032899 time 760813992308 state 0x0 ref 1 00000100:00100000:1.0:1789760776.534360:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760814252724 time 760813992308 state 0x1 ref 1 00000100:00100000:1.0:1789760776.534387:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760814252724 time 760855103961 state 0x0 ref 1 00000100:00100000:1.0:1789760776.534400:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495909504:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760776.534447:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26447872, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd009 00010000:00010000:1.0:1789760776.534457:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x493674835454785b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.534472:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760776.534477:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.534484:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x493674835454785b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.534499:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x493674835454785b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.562621:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973774 is committed 00002000:00010000:3.0:1789760776.562952:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26447872 -> 26451968 00010000:00010000:3.0:1789760776.562977:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x493674835454785b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.562997:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x493674835454785b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.563007:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x493674835454785b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.563014:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760776.563019:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x493674835454785b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.563028:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x493674835454785b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.563050:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x493674835454785b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.563058:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x493674835454785b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.563150:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495909504:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28748us (28999us total) trans 4294973774 rc 0/0 00000100:00100000:3.0:1789760776.563160:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6486 00000100:00100000:3.0:1789760776.563170:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760814252724 time 760855103961 state 0x2 ref 0 00000100:00100000:0.0:1789760776.566989:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760776.567133:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495909632 00000100:00100000:2.0:1789760776.567144:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495909632 00000100:00100000:2.0:1789760776.567209:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760814252724 time 760855103961 state 0x0 ref 1 00000100:00100000:2.0:1789760776.567225:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760855364377 time 760855103961 state 0x1 ref 1 00000100:00100000:2.0:1789760776.567256:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760855364377 time 760887972865 state 0x0 ref 1 00000100:00100000:2.0:1789760776.567270:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495909632:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760776.567359:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26451968, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd009 00010000:00010000:2.0:1789760776.567369:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547862 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.567385:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760776.567390:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.567396:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547862 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.567410:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547862 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.603670:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973775 is committed 00002000:00010000:0.0:1789760776.603932:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26451968 -> 26456064 00010000:00010000:0.0:1789760776.603948:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547862 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.603961:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547862 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.603969:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547862 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.603985:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760776.603990:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547862 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.603999:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547862 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.604026:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547862 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.604034:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547862 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.604162:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495909632:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36890us (37188us total) trans 4294973775 rc 0/0 00000100:00100000:0.0:1789760776.604172:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6487 00000100:00100000:0.0:1789760776.604182:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760855364377 time 760887972865 state 0x2 ref 0 00000100:00100000:2.0:1789760776.608710:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760776.609569:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495909760 00000100:00100000:3.0:1789760776.609582:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495909760 00000100:00100000:3.0:1789760776.609930:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760855364377 time 760887972865 state 0x0 ref 1 00000100:00100000:3.0:1789760776.609946:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760888233281 time 760887972865 state 0x1 ref 1 00000100:00100000:3.0:1789760776.610032:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760888233281 time 760930748713 state 0x0 ref 1 00000100:00100000:3.0:1789760776.610047:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495909760:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760776.610215:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26456064, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd009 00010000:00010000:3.0:1789760776.610225:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547869 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.610240:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760776.610244:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.610251:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547869 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.610264:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547869 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760776.651709:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26456064 -> 26460160 00010000:00010000:0.0:1789760776.651729:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547869 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.651743:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547869 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.651751:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547869 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.651758:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760776.651762:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547869 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.651846:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547869 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.651872:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547869 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.651880:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547869 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.651979:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495909760:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41931us (43275us total) trans 4294973776 rc 0/0 00000100:00100000:0.0:1789760776.651989:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6488 00000100:00100000:0.0:1789760776.652000:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760888233281 time 760930748713 state 0x2 ref 0 00000001:00080000:3.0:1789760776.654234:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973776 is committed 00000100:00100000:2.0:1789760776.657772:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760776.657873:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495909888 00000100:00100000:3.0:1789760776.657884:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495909888 00000100:00100000:3.0:1789760776.657938:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760888233281 time 760930748713 state 0x0 ref 1 00000100:00100000:3.0:1789760776.657965:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760931009129 time 760930748713 state 0x1 ref 1 00000100:00100000:3.0:1789760776.658038:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760931009129 time 760978754662 state 0x0 ref 1 00000100:00100000:3.0:1789760776.658052:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495909888:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760776.658103:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26460160, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd009 00010000:00010000:3.0:1789760776.658113:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547870 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.658135:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760776.658139:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.658146:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547870 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.658164:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547870 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.693629:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973777 is committed 00002000:00010000:0.0:1789760776.693901:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26460160 -> 26464256 00002000:00010000:0.0:1789760776.693905:0:13059:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760775 -> 1789760776 00002000:00010000:0.0:1789760776.693908:0:13059:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760775 -> 1789760776 00010000:00010000:0.0:1789760776.693922:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547870 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.693933:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547870 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.693943:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547870 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.693950:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760776.693956:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547870 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.693966:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547870 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.693987:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547870 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.693996:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547870 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.694691:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495909888:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36633us (36919us total) trans 4294973777 rc 0/0 00000100:00100000:0.0:1789760776.694702:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6489 00000100:00100000:0.0:1789760776.694713:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760931009129 time 760978754662 state 0x2 ref 0 00000100:00100000:0.0:1789760776.698897:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760776.699063:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495910016 00000100:00100000:2.0:1789760776.699075:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495910016 00000100:00100000:2.0:1789760776.699130:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760931009129 time 760978754662 state 0x0 ref 1 00000100:00100000:2.0:1789760776.699143:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760979015078 time 760978754662 state 0x1 ref 1 00000100:00100000:2.0:1789760776.699174:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760979015078 time 761019891212 state 0x0 ref 1 00000100:00100000:2.0:1789760776.699187:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495910016:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760776.699232:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26464256, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd009 00010000:00010000:2.0:1789760776.699239:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547877 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.699251:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760776.699255:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.699259:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547877 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.699268:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547877 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.731238:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973778 is committed 00002000:00010000:1.0:1789760776.731386:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26464256 -> 26468352 00010000:00010000:1.0:1789760776.731401:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547877 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.731415:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547877 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.731425:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547877 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.731437:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760776.731442:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547877 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.731450:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547877 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.731471:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547877 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.731479:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547877 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.731677:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495910016:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32487us (32785us total) trans 4294973778 rc 0/0 00000100:00100000:1.0:1789760776.731688:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6490 00000100:00100000:1.0:1789760776.731698:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760979015078 time 761019891212 state 0x2 ref 0 00000100:00100000:3.0:1789760776.738873:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760776.742515:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495910144 00000100:00100000:0.0:1789760776.742529:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495910144 00000100:00100000:0.0:1789760776.742592:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 760979015078 time 761019891212 state 0x0 ref 1 00000100:00100000:0.0:1789760776.742608:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761020151628 time 761019891212 state 0x1 ref 1 00000100:00100000:0.0:1789760776.742639:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761020151628 time 761063355307 state 0x0 ref 1 00000100:00100000:0.0:1789760776.742652:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495910144:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760776.742700:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26468352, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd009 00010000:00010000:0.0:1789760776.742709:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454787e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.742725:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760776.742730:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.742736:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454787e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.742750:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454787e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760776.773273:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26468352 -> 26472448 00010000:00010000:0.0:1789760776.773316:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454787e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.773330:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454787e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.773335:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973779 is committed 00010000:00010000:0.0:1789760776.773339:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454787e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.773345:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760776.773350:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454787e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.773359:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454787e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.773384:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454787e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.773392:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x493674835454787e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.773483:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495910144:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30829us (34614us total) trans 4294973779 rc 0/0 00000100:00100000:0.0:1789760776.773493:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6491 00000100:00100000:0.0:1789760776.773504:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761020151628 time 761063355307 state 0x2 ref 0 00000100:00100000:0.0:1789760776.779815:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760776.779993:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495910272 00000100:00100000:3.0:1789760776.780003:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495910272 00000100:00100000:3.0:1789760776.780055:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761020151628 time 761063355307 state 0x0 ref 1 00000100:00100000:3.0:1789760776.780070:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761063615723 time 761063355307 state 0x1 ref 1 00000100:00100000:3.0:1789760776.780141:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761063615723 time 761100856148 state 0x0 ref 1 00000100:00100000:3.0:1789760776.780155:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495910272:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760776.780204:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26472448, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd009 00010000:00010000:3.0:1789760776.780214:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547885 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.780230:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760776.780235:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.780241:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547885 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.780256:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547885 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760776.811650:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26472448 -> 26476544 00010000:00010000:2.0:1789760776.811673:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547885 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.811687:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547885 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.811696:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547885 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.811702:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760776.811708:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547885 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.811717:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547885 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.811736:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547885 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.811743:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547885 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.811840:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495910272:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31682us (32030us total) trans 4294973780 rc 0/0 00000100:00100000:2.0:1789760776.811851:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6492 00000100:00100000:2.0:1789760776.811860:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761063615723 time 761100856148 state 0x2 ref 0 00000001:00080000:3.0:1789760776.811926:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973780 is committed 00000100:00100000:2.0:1789760776.816333:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760776.816517:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495910400 00000100:00100000:1.0:1789760776.816526:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495910400 00000100:00100000:1.0:1789760776.816580:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761063615723 time 761100856148 state 0x0 ref 1 00000100:00100000:1.0:1789760776.816596:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761101116564 time 761100856148 state 0x1 ref 1 00000100:00100000:1.0:1789760776.816621:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761101116564 time 761137338214 state 0x0 ref 1 00000100:00100000:1.0:1789760776.816637:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495910400:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760776.816683:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26476544, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd009 00010000:00010000:1.0:1789760776.816693:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454788c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.816709:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760776.816714:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.816721:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454788c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.816746:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454788c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.844137:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973781 is committed 00002000:00010000:2.0:1789760776.844326:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26476544 -> 26480640 00010000:00010000:2.0:1789760776.844341:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454788c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.844354:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454788c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.844373:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454788c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.844381:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760776.844386:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454788c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.844396:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454788c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.844415:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454788c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.844423:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454788c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.844708:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495910400:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28068us (28379us total) trans 4294973781 rc 0/0 00000100:00100000:2.0:1789760776.844723:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6493 00000100:00100000:2.0:1789760776.844740:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761101116564 time 761137338214 state 0x2 ref 0 00000100:00100000:1.0:1789760776.854336:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760776.854432:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495910528 00000100:00100000:0.0:1789760776.854442:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495910528 00000100:00100000:0.0:1789760776.854499:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761101116564 time 761137338214 state 0x0 ref 1 00000100:00100000:0.0:1789760776.854516:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761137598630 time 761137338214 state 0x1 ref 1 00000100:00100000:0.0:1789760776.854584:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761137598630 time 761175299662 state 0x0 ref 1 00000100:00100000:0.0:1789760776.854600:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495910528:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760776.854652:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26480640, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd009 00010000:00010000:0.0:1789760776.854662:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547893 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.854680:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760776.854684:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.854690:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547893 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.854703:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547893 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.908841:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973782 is committed 00002000:00010000:2.0:1789760776.908952:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26480640 -> 26484736 00010000:00010000:2.0:1789760776.908967:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547893 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.908980:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547893 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.908989:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547893 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.908996:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760776.909001:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547893 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.909009:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547893 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.909028:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547893 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.909036:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547893 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.909127:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495910528:12345-192.168.203.9@tcp:4:dd.0 Request processed in 54526us (54796us total) trans 4294973782 rc 0/0 00000100:00100000:2.0:1789760776.909136:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6494 00000100:00100000:2.0:1789760776.909146:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761137598630 time 761175299662 state 0x2 ref 0 00000100:00100000:3.0:1789760776.914813:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760776.915766:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495910656 00000100:00100000:0.0:1789760776.915787:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495910656 00000100:00100000:0.0:1789760776.915842:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761137598630 time 761175299662 state 0x0 ref 1 00000100:00100000:0.0:1789760776.915857:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761175560078 time 761175299662 state 0x1 ref 1 00000100:00100000:0.0:1789760776.915930:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761175560078 time 761236646328 state 0x0 ref 1 00000100:00100000:0.0:1789760776.915945:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f872680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495910656:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760776.916001:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26484736, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd009 00010000:00010000:0.0:1789760776.916010:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454789a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.916025:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760776.916030:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.916037:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454789a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.916053:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454789a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.964344:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973783 is committed 00002000:00010000:3.0:1789760776.964477:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26484736 -> 26488832 00010000:00010000:3.0:1789760776.964493:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454789a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.964506:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454789a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.964515:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454789a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.964522:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760776.964528:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454789a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.964537:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454789a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.964557:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454789a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.964578:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x493674835454789a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.964674:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f872680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495910656:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48727us (49865us total) trans 4294973783 rc 0/0 00000100:00100000:3.0:1789760776.964684:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6495 00000100:00100000:3.0:1789760776.964695:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761175560078 time 761236646328 state 0x2 ref 0 00000100:00100000:3.0:1789760776.967754:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760776.967840:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495910784 00000100:00100000:2.0:1789760776.967849:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495910784 00000100:00100000:2.0:1789760776.967900:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761175560078 time 761236646328 state 0x0 ref 1 00000100:00100000:2.0:1789760776.967914:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761236906744 time 761236646328 state 0x1 ref 1 00000100:00100000:2.0:1789760776.967991:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761236906744 time 761288707209 state 0x0 ref 1 00000100:00100000:2.0:1789760776.968004:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495910784:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760776.968052:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26488832, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd009 00010000:00010000:2.0:1789760776.968060:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545478a1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.968086:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760776.968092:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.968098:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545478a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.968111:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545478a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760777.013981:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26488832 -> 26492928 00010000:00010000:1.0:1789760777.014006:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545478a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.014021:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545478a1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.014030:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545478a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.014037:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760777.014042:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545478a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.014052:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545478a1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.014071:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545478a1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.014078:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545478a1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.014212:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495910784:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46206us (46461us total) trans 4294973784 rc 0/0 00000100:00100000:1.0:1789760777.014224:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6496 00000100:00100000:1.0:1789760777.014234:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761236906744 time 761288707209 state 0x2 ref 0 00000001:00080000:1.0:1789760777.014314:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973784 is committed 00000100:00100000:1.0:1789760777.018277:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760777.018384:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495910912 00000100:00100000:2.0:1789760777.018394:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495910912 00000100:00080000:1.0:1789760777.018441:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760777.018469:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761236906744 time 761288707209 state 0x0 ref 1 00000100:00100000:2.0:1789760777.018485:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761288967625 time 761288707209 state 0x1 ref 1 00000100:00100000:2.0:1789760777.018517:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761288967625 time 761339233545 state 0x0 ref 1 00000100:00100000:2.0:1789760777.018529:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495910912:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760777.018575:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26492928, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd009 00010000:00010000:2.0:1789760777.018583:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545478a8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.018599:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760777.018603:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.018609:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545478a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.018621:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545478a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.057581:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760777.068169:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973785 is committed 00002000:00010000:0.0:1789760777.068336:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26492928 -> 26497024 00010000:00010000:0.0:1789760777.068352:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545478a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.068364:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545478a8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.068374:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545478a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.068383:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760777.068387:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545478a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.068395:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545478a8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.068413:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545478a8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.068421:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545478a8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.068518:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495910912:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49986us (50245us total) trans 4294973785 rc 0/0 00000100:00100000:0.0:1789760777.068527:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6497 00000100:00100000:0.0:1789760777.068536:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761288967625 time 761339233545 state 0x2 ref 0 00000100:00100000:3.0:1789760777.076176:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760777.076349:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495911040 00000100:00100000:1.0:1789760777.076360:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495911040 00000100:00100000:1.0:1789760777.076413:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761288967625 time 761339233545 state 0x0 ref 1 00000100:00100000:1.0:1789760777.076427:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761339493961 time 761339233545 state 0x1 ref 1 00000100:00100000:1.0:1789760777.076459:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761339493961 time 761397175714 state 0x0 ref 1 00000100:00100000:1.0:1789760777.076473:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495911040:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760777.076518:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26497024, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd009 00010000:00010000:1.0:1789760777.076526:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545478af lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.076542:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760777.076546:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.076552:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545478af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.076566:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545478af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.118825:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973786 is committed 00002000:00010000:2.0:1789760777.118929:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26497024 -> 26501120 00010000:00010000:2.0:1789760777.118945:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545478af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.118959:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545478af lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.119039:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545478af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.119048:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760777.119053:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545478af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.119062:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545478af lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.119082:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545478af lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.119091:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x49367483545478af lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.119219:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495911040:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42742us (43048us total) trans 4294973786 rc 0/0 00000100:00100000:2.0:1789760777.119230:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6498 00000100:00100000:2.0:1789760777.119240:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761339493961 time 761397175714 state 0x2 ref 0 00000100:00100000:3.0:1789760777.122935:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760777.123066:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495911168 00000100:00100000:1.0:1789760777.123077:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495911168 00000100:00100000:1.0:1789760777.123141:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761339493961 time 761397175714 state 0x0 ref 1 00000100:00100000:1.0:1789760777.123156:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761397436130 time 761397175714 state 0x1 ref 1 00000100:00100000:1.0:1789760777.123272:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761397436130 time 761443987997 state 0x0 ref 1 00000100:00100000:1.0:1789760777.123310:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495911168:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760777.123359:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26501120, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd009 00010000:00010000:1.0:1789760777.123368:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545478b6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.123386:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760777.123391:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.123397:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545478b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.123411:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545478b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.154065:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973787 is committed 00002000:00010000:2.0:1789760777.154168:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26501120 -> 26505216 00010000:00010000:2.0:1789760777.154184:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545478b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.154208:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545478b6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.154218:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545478b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.154226:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760777.154232:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545478b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.154242:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545478b6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.154263:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545478b6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.154270:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545478b6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.154392:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495911168:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31080us (31461us total) trans 4294973787 rc 0/0 00000100:00100000:2.0:1789760777.154402:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6499 00000100:00100000:2.0:1789760777.154412:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761397436130 time 761443987997 state 0x2 ref 0 00000100:00100000:1.0:1789760777.158840:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760777.159010:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495911296 00000100:00100000:0.0:1789760777.159020:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495911296 00000100:00100000:0.0:1789760777.159076:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761397436130 time 761443987997 state 0x0 ref 1 00000100:00100000:0.0:1789760777.159092:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761444248413 time 761443987997 state 0x1 ref 1 00000100:00100000:0.0:1789760777.159180:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761444248413 time 761479895829 state 0x0 ref 1 00000100:00100000:0.0:1789760777.159193:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495911296:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760777.159239:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26505216, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd009 00010000:00010000:0.0:1789760777.159248:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545478bd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.159264:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760777.159269:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.159276:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545478bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.159310:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545478bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.181662:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973788 is committed 00002000:00010000:0.0:1789760777.181794:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26505216 -> 26509312 00010000:00010000:0.0:1789760777.181812:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545478bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.181826:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545478bd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.181835:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545478bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.181842:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760777.181847:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545478bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.181855:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545478bd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.181874:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545478bd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.181882:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545478bd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.181975:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495911296:12345-192.168.203.9@tcp:4:dd.0 Request processed in 22778us (23138us total) trans 4294973788 rc 0/0 00000100:00100000:0.0:1789760777.181989:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6500 00000100:00100000:0.0:1789760777.181999:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761444248413 time 761479895829 state 0x2 ref 0 00000100:00100000:3.0:1789760777.188080:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760777.188180:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495911424 00000100:00100000:2.0:1789760777.188189:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495911424 00000100:00100000:2.0:1789760777.188242:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761444248413 time 761479895829 state 0x0 ref 1 00000100:00100000:2.0:1789760777.188256:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761480156245 time 761479895829 state 0x1 ref 1 00000100:00100000:2.0:1789760777.188317:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761480156245 time 761509034276 state 0x0 ref 1 00000100:00100000:2.0:1789760777.188329:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495911424:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760777.188373:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26509312, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd009 00010000:00010000:2.0:1789760777.188381:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545478c4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.188397:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760777.188401:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.188407:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545478c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.188419:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545478c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.222227:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973789 is committed 00002000:00010000:2.0:1789760777.222303:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26509312 -> 26513408 00010000:00010000:2.0:1789760777.222319:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545478c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.222338:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545478c4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.222348:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545478c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.222355:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760777.222360:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545478c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.222368:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545478c4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.222388:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545478c4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.222395:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545478c4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.222482:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495911424:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34149us (34408us total) trans 4294973789 rc 0/0 00000100:00100000:2.0:1789760777.222492:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6501 00000100:00100000:2.0:1789760777.222502:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761480156245 time 761509034276 state 0x2 ref 0 00000100:00100000:3.0:1789760777.227451:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760777.227873:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495911552 00000100:00100000:2.0:1789760777.227883:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495911552 00000100:00100000:2.0:1789760777.227939:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761480156245 time 761509034276 state 0x0 ref 1 00000100:00100000:2.0:1789760777.227954:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761509294692 time 761509034276 state 0x1 ref 1 00000100:00100000:2.0:1789760777.228419:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761509294692 time 761549132384 state 0x0 ref 1 00000100:00100000:2.0:1789760777.228435:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495911552:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760777.228483:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26513408, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd009 00010000:00010000:2.0:1789760777.228492:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478cb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.228508:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760777.228514:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.228520:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.228534:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.258644:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973790 is committed 00002000:00010000:2.0:1789760777.258730:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26513408 -> 26517504 00010000:00010000:2.0:1789760777.258747:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.258841:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478cb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.258856:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.258864:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760777.258869:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.258878:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478cb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.258898:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478cb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.258907:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478cb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.259042:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495911552:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30605us (31594us total) trans 4294973790 rc 0/0 00000100:00100000:2.0:1789760777.259052:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6502 00000100:00100000:2.0:1789760777.259063:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761509294692 time 761549132384 state 0x2 ref 0 00000100:00100000:1.0:1789760777.262219:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760777.262655:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495911680 00000100:00100000:0.0:1789760777.262663:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495911680 00000100:00100000:0.0:1789760777.262710:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761509294692 time 761549132384 state 0x0 ref 1 00000100:00100000:0.0:1789760777.262721:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761549392800 time 761549132384 state 0x1 ref 1 00000100:00100000:0.0:1789760777.262785:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761549392800 time 761583490657 state 0x0 ref 1 00000100:00100000:0.0:1789760777.262798:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495911680:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760777.262851:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26517504, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd009 00010000:00010000:0.0:1789760777.262923:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545478d2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.262935:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760777.262939:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.262943:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545478d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.262954:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545478d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.284705:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973791 is committed 00002000:00010000:2.0:1789760777.285228:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26517504 -> 26521600 00010000:00010000:2.0:1789760777.285242:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545478d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.285256:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545478d2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.285262:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545478d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.285266:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760777.285269:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545478d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.285275:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545478d2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.285319:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545478d2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.285327:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545478d2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.285419:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495911680:12345-192.168.203.9@tcp:4:dd.0 Request processed in 22618us (23202us total) trans 4294973791 rc 0/0 00000100:00100000:2.0:1789760777.285429:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6503 00000100:00100000:2.0:1789760777.285438:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761549392800 time 761583490657 state 0x2 ref 0 00000100:00100000:3.0:1789760777.289140:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760777.289376:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495911808 00000100:00100000:2.0:1789760777.289387:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495911808 00000100:00100000:2.0:1789760777.289445:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761549392800 time 761583490657 state 0x0 ref 1 00000100:00100000:2.0:1789760777.289460:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761583751073 time 761583490657 state 0x1 ref 1 00000100:00100000:2.0:1789760777.289494:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761583751073 time 761610210535 state 0x0 ref 1 00000100:00100000:2.0:1789760777.289507:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495911808:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760777.289554:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26521600, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd009 00010000:00010000:2.0:1789760777.289563:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478d9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.289579:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760777.289584:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.289591:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.289604:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.324969:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973792 is committed 00002000:00010000:2.0:1789760777.325226:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26521600 -> 26525696 00010000:00010000:2.0:1789760777.325242:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.325255:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478d9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.325270:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.325277:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760777.325306:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.325317:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478d9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.325339:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478d9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.325347:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478d9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.325514:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495911808:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36005us (36378us total) trans 4294973792 rc 0/0 00000100:00100000:2.0:1789760777.325524:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6504 00000100:00100000:2.0:1789760777.325534:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761583751073 time 761610210535 state 0x2 ref 0 00000100:00100000:3.0:1789760777.329694:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760777.329783:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495911936 00000100:00100000:2.0:1789760777.329793:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495911936 00000100:00100000:2.0:1789760777.329845:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761583751073 time 761610210535 state 0x0 ref 1 00000100:00100000:2.0:1789760777.329861:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761610470951 time 761610210535 state 0x1 ref 1 00000100:00100000:2.0:1789760777.329892:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761610470951 time 761650609320 state 0x0 ref 1 00000100:00100000:2.0:1789760777.329904:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fdca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495911936:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760777.329955:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26525696, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd009 00010000:00010000:2.0:1789760777.329969:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545478e0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.329984:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760777.329989:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.329995:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545478e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.330008:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545478e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.359151:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973793 is committed 00002000:00010000:3.0:1789760777.359257:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26525696 -> 26529792 00010000:00010000:3.0:1789760777.359270:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545478e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.359306:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545478e0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.359311:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545478e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.359315:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760777.359319:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545478e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.359325:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545478e0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.359340:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545478e0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.359344:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x49367483545478e0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.359416:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fdca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495911936:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29509us (29726us total) trans 4294973793 rc 0/0 00000100:00100000:3.0:1789760777.359427:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6505 00000100:00100000:3.0:1789760777.359436:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761610470951 time 761650609320 state 0x2 ref 0 00000100:00100000:1.0:1789760777.362178:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760777.362238:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495912064 00000100:00100000:2.0:1789760777.362248:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495912064 00000100:00100000:2.0:1789760777.362332:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761610470951 time 761650609320 state 0x0 ref 1 00000100:00100000:2.0:1789760777.362347:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761650869736 time 761650609320 state 0x1 ref 1 00000100:00100000:2.0:1789760777.362377:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761650869736 time 761683093469 state 0x0 ref 1 00000100:00100000:2.0:1789760777.362390:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495912064:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760777.362432:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26529792, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd009 00010000:00010000:2.0:1789760777.362442:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478e7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.362458:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760777.362463:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.362469:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.362483:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.388392:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973794 is committed 00002000:00010000:2.0:1789760777.391250:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26529792 -> 26533888 00010000:00010000:2.0:1789760777.391270:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.391300:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478e7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.391308:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.391316:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760777.391320:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.391330:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478e7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.391355:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478e7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.391364:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478e7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.391461:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495912064:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29069us (29284us total) trans 4294973794 rc 0/0 00000100:00100000:2.0:1789760777.391471:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6506 00000100:00100000:2.0:1789760777.391481:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761650869736 time 761683093469 state 0x2 ref 0 00000100:00100000:0.0:1789760777.395482:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760777.395837:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495912192 00000100:00100000:2.0:1789760777.395847:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495912192 00000100:00100000:2.0:1789760777.395900:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761650869736 time 761683093469 state 0x0 ref 1 00000100:00100000:2.0:1789760777.395915:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761683353885 time 761683093469 state 0x1 ref 1 00000100:00100000:2.0:1789760777.395948:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761683353885 time 761716664987 state 0x0 ref 1 00000100:00100000:2.0:1789760777.395959:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495912192:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760777.396006:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26533888, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd009 00010000:00010000:2.0:1789760777.396015:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545478ee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.396031:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760777.396036:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.396043:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545478ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.396055:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545478ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.422520:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973795 is committed 00002000:00010000:3.0:1789760777.422621:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26533888 -> 26537984 00010000:00010000:3.0:1789760777.422637:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545478ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.422650:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545478ee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.422658:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545478ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.422666:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760777.422671:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545478ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.422680:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545478ee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.422698:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545478ee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.422705:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545478ee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.422810:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495912192:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26848us (27334us total) trans 4294973795 rc 0/0 00000100:00100000:3.0:1789760777.422820:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6507 00000100:00100000:3.0:1789760777.422830:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761683353885 time 761716664987 state 0x2 ref 0 00000100:00100000:2.0:1789760777.427033:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760777.427124:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495912320 00000100:00100000:3.0:1789760777.427134:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495912320 00000100:00100000:3.0:1789760777.427196:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761683353885 time 761716664987 state 0x0 ref 1 00000100:00100000:3.0:1789760777.427213:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761716925403 time 761716664987 state 0x1 ref 1 00000100:00100000:3.0:1789760777.427246:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761716925403 time 761747963373 state 0x0 ref 1 00000100:00100000:3.0:1789760777.427258:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495912320:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760777.427344:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26537984, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd009 00010000:00010000:3.0:1789760777.427354:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545478f5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.427370:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760777.427373:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.427379:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545478f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.427392:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545478f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760777.463769:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26537984 -> 26542080 00010000:00010000:3.0:1789760777.463800:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545478f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.463813:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545478f5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.463823:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545478f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.463831:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760777.463836:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545478f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.463845:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545478f5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.463865:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545478f5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.463873:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545478f5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.463968:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495912320:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36707us (36938us total) trans 4294973796 rc 0/0 00000100:00100000:3.0:1789760777.463978:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6508 00000100:00100000:3.0:1789760777.463988:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761716925403 time 761747963373 state 0x2 ref 0 00000001:00080000:0.0:1789760777.465585:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973796 is committed 00000100:00100000:1.0:1789760777.471353:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760777.473960:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495912448 00000100:00100000:2.0:1789760777.473974:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495912448 00000100:00100000:2.0:1789760777.474036:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761716925403 time 761747963373 state 0x0 ref 1 00000100:00100000:2.0:1789760777.474052:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761748223789 time 761747963373 state 0x1 ref 1 00000100:00100000:2.0:1789760777.474170:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761748223789 time 761794875617 state 0x0 ref 1 00000100:00100000:2.0:1789760777.474186:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495912448:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760777.474237:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26542080, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd009 00010000:00010000:2.0:1789760777.474247:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478fc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.474263:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760777.474269:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.474276:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.474312:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.500526:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973797 is committed 00002000:00010000:2.0:1789760777.500623:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26542080 -> 26546176 00010000:00010000:2.0:1789760777.500639:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.500653:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478fc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.500662:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.500669:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760777.500673:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.500682:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.500702:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478fc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.500710:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x49367483545478fc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.500820:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495912448:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26633us (29471us total) trans 4294973797 rc 0/0 00000100:00100000:2.0:1789760777.500833:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6509 00000100:00100000:2.0:1789760777.500843:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761748223789 time 761794875617 state 0x2 ref 0 00000100:00100000:1.0:1789760777.506672:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760777.507653:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495912576 00000100:00100000:3.0:1789760777.507665:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495912576 00000100:00100000:3.0:1789760777.507723:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761748223789 time 761794875617 state 0x0 ref 1 00000100:00100000:3.0:1789760777.507740:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761795136033 time 761794875617 state 0x1 ref 1 00000100:00100000:3.0:1789760777.507782:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761795136033 time 761828491017 state 0x0 ref 1 00000100:00100000:3.0:1789760777.507795:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495912576:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760777.507842:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26546176, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd009 00010000:00010000:3.0:1789760777.507852:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547903 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.507869:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760777.507874:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.507891:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547903 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.507906:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547903 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.534201:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973798 is committed 00002000:00010000:0.0:1789760777.534432:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26546176 -> 26550272 00010000:00010000:0.0:1789760777.534448:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547903 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.534461:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547903 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.534471:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547903 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.534478:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760777.534483:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547903 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.534492:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547903 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.534511:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547903 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.534518:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547903 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.534619:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495912576:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26822us (27952us total) trans 4294973798 rc 0/0 00000100:00100000:0.0:1789760777.534631:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6510 00000100:00100000:0.0:1789760777.534641:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761795136033 time 761828491017 state 0x2 ref 0 00000100:00100000:1.0:1789760777.541258:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760777.541333:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495912704 00000100:00100000:3.0:1789760777.541343:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495912704 00000100:00100000:3.0:1789760777.541398:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761795136033 time 761828491017 state 0x0 ref 1 00000100:00100000:3.0:1789760777.541413:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761828751433 time 761828491017 state 0x1 ref 1 00000100:00100000:3.0:1789760777.541442:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761828751433 time 761862158786 state 0x0 ref 1 00000100:00100000:3.0:1789760777.541455:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495912704:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760777.541507:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26550272, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd009 00010000:00010000:3.0:1789760777.541517:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454790a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.541533:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760777.541537:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.541544:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454790a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.541558:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454790a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760777.579475:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26550272 -> 26554368 00010000:00010000:0.0:1789760777.579495:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454790a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.579509:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454790a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.579518:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454790a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.579525:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760777.579531:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454790a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.579540:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454790a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.579558:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454790a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.579571:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454790a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.579660:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495912704:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38204us (38405us total) trans 4294973799 rc 0/0 00000100:00100000:0.0:1789760777.579671:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6511 00000100:00100000:0.0:1789760777.579682:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761828751433 time 761862158786 state 0x2 ref 0 00000001:00080000:1.0:1789760777.582709:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973799 is committed 00000100:00100000:1.0:1789760777.584197:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760777.584523:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495912832 00000100:00100000:2.0:1789760777.584533:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495912832 00000100:00100000:2.0:1789760777.584586:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761828751433 time 761862158786 state 0x0 ref 1 00000100:00100000:2.0:1789760777.584600:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761862419202 time 761862158786 state 0x1 ref 1 00000100:00100000:2.0:1789760777.584630:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761862419202 time 761905347518 state 0x0 ref 1 00000100:00100000:2.0:1789760777.584643:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495912832:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760777.584688:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26554368, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd009 00010000:00010000:2.0:1789760777.584698:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547911 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.584714:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760777.584719:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.584725:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547911 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.584738:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547911 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760777.610534:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26554368 -> 26558464 00010000:00010000:0.0:1789760777.610552:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547911 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.610566:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547911 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.610575:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547911 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.610582:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760777.610587:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547911 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.610595:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547911 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.610619:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547911 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.610627:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547911 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.610712:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495912832:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26066us (26519us total) trans 4294973800 rc 0/0 00000100:00100000:0.0:1789760777.610721:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6512 00000100:00100000:0.0:1789760777.610730:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761862419202 time 761905347518 state 0x2 ref 0 00000001:00080000:1.0:1789760777.610858:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973800 is committed 00000100:00100000:3.0:1789760777.614339:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760777.614532:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495912960 00000100:00100000:0.0:1789760777.614541:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495912960 00000100:00100000:0.0:1789760777.614592:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761862419202 time 761905347518 state 0x0 ref 1 00000100:00100000:0.0:1789760777.614606:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761905607934 time 761905347518 state 0x1 ref 1 00000100:00100000:0.0:1789760777.614632:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761905607934 time 761935348860 state 0x0 ref 1 00000100:00100000:0.0:1789760777.614644:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495912960:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760777.614689:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26558464, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd009 00010000:00010000:0.0:1789760777.614697:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547918 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.614713:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760777.614717:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.614723:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547918 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.614736:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547918 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.643780:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973801 is committed 00002000:00010000:2.0:1789760777.643995:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26558464 -> 26562560 00010000:00010000:2.0:1789760777.644010:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547918 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.644023:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547918 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.644032:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547918 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.644040:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760777.644045:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547918 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.644053:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547918 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.644072:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547918 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.644081:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547918 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.644349:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495912960:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29703us (30014us total) trans 4294973801 rc 0/0 00000100:00100000:2.0:1789760777.644360:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6513 00000100:00100000:2.0:1789760777.644369:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761905607934 time 761935348860 state 0x2 ref 0 00000100:00100000:3.0:1789760777.647409:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760777.647492:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495913088 00000100:00100000:0.0:1789760777.647501:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495913088 00000100:00100000:0.0:1789760777.647548:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761905607934 time 761935348860 state 0x0 ref 1 00000100:00100000:0.0:1789760777.647562:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761935609276 time 761935348860 state 0x1 ref 1 00000100:00100000:0.0:1789760777.647588:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761935609276 time 761968306117 state 0x0 ref 1 00000100:00100000:0.0:1789760777.647599:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495913088:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760777.647639:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26562560, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd009 00010000:00010000:0.0:1789760777.647648:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454791f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.647662:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760777.647667:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.647673:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454791f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.647687:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454791f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760777.675712:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26562560 -> 26566656 00002000:00010000:0.0:1789760777.675720:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760776 -> 1789760777 00002000:00010000:0.0:1789760777.675723:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760776 -> 1789760777 00010000:00010000:0.0:1789760777.675736:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454791f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.675758:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454791f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.675768:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454791f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.675784:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760777.675789:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454791f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.675798:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454791f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.675818:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454791f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.675826:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x493674835454791f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.675924:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495913088:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28321us (28518us total) trans 4294973802 rc 0/0 00000100:00100000:0.0:1789760777.675934:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6514 00000100:00100000:0.0:1789760777.675944:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761935609276 time 761968306117 state 0x2 ref 0 00000001:00080000:2.0:1789760777.677875:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973802 is committed 00000100:00100000:1.0:1789760777.680536:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760777.680713:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495913216 00000100:00100000:2.0:1789760777.680722:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495913216 00000100:00100000:2.0:1789760777.680790:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761935609276 time 761968306117 state 0x0 ref 1 00000100:00100000:2.0:1789760777.680804:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761968566533 time 761968306117 state 0x1 ref 1 00000100:00100000:2.0:1789760777.680830:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761968566533 time 762001547269 state 0x0 ref 1 00000100:00100000:2.0:1789760777.680841:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495913216:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760777.680885:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26566656, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd009 00010000:00010000:2.0:1789760777.680893:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547926 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.680909:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760777.680913:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.680919:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547926 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.680982:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547926 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.706532:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973803 is committed 00002000:00010000:0.0:1789760777.706673:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26566656 -> 26570752 00010000:00010000:0.0:1789760777.706689:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547926 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.706704:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547926 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.706713:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547926 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.706721:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760777.706726:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547926 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.706735:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547926 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.706768:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547926 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.706789:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547926 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.706876:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495913216:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26033us (26346us total) trans 4294973803 rc 0/0 00000100:00100000:0.0:1789760777.706887:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6515 00000100:00100000:0.0:1789760777.706897:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761968566533 time 762001547269 state 0x2 ref 0 00000100:00100000:1.0:1789760777.713602:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760777.713725:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495913344 00000100:00100000:2.0:1789760777.713736:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495913344 00000100:00100000:2.0:1789760777.713811:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 761968566533 time 762001547269 state 0x0 ref 1 00000100:00100000:2.0:1789760777.713828:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762001807685 time 762001547269 state 0x1 ref 1 00000100:00100000:2.0:1789760777.713892:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762001807685 time 762034608248 state 0x0 ref 1 00000100:00100000:2.0:1789760777.713905:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495913344:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760777.713954:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26570752, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd009 00010000:00010000:2.0:1789760777.713963:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454792d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.713979:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760777.713983:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.713990:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454792d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.714007:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454792d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.762413:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973804 is committed 00002000:00010000:2.0:1789760777.762473:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26570752 -> 26574848 00010000:00010000:2.0:1789760777.762487:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454792d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.762500:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454792d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.762508:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454792d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.762519:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760777.762524:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454792d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.762532:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454792d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.762550:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454792d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.762558:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x493674835454792d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.762651:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495913344:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48743us (49053us total) trans 4294973804 rc 0/0 00000100:00100000:2.0:1789760777.762660:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6516 00000100:00100000:2.0:1789760777.762669:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762001807685 time 762034608248 state 0x2 ref 0 00000100:00100000:3.0:1789760777.769657:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760777.769923:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495913472 00000100:00100000:0.0:1789760777.769933:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495913472 00000100:00100000:0.0:1789760777.769989:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762001807685 time 762034608248 state 0x0 ref 1 00000100:00100000:0.0:1789760777.770004:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762034868664 time 762034608248 state 0x1 ref 1 00000100:00100000:0.0:1789760777.770031:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762034868664 time 762090748006 state 0x0 ref 1 00000100:00100000:0.0:1789760777.770044:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495913472:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760777.770090:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26574848, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd009 00010000:00010000:0.0:1789760777.770099:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547934 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.770114:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760777.770119:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.770125:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547934 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.770144:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547934 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.825623:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973805 is committed 00002000:00010000:0.0:1789760777.825728:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26574848 -> 26578944 00010000:00010000:0.0:1789760777.825744:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547934 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.825765:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547934 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.825789:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547934 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.825797:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760777.825802:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547934 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.825812:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547934 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.825831:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547934 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.825838:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547934 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.825954:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495913472:12345-192.168.203.9@tcp:4:dd.0 Request processed in 55908us (56302us total) trans 4294973805 rc 0/0 00000100:00100000:0.0:1789760777.825964:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6517 00000100:00100000:0.0:1789760777.825973:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762034868664 time 762090748006 state 0x2 ref 0 00000100:00100000:0.0:1789760777.830578:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760777.830683:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495913600 00000100:00100000:2.0:1789760777.830693:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495913600 00000100:00100000:2.0:1789760777.830810:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762034868664 time 762090748006 state 0x0 ref 1 00000100:00100000:2.0:1789760777.830829:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762091008422 time 762090748006 state 0x1 ref 1 00000100:00100000:2.0:1789760777.830906:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762091008422 time 762151621785 state 0x0 ref 1 00000100:00100000:2.0:1789760777.830919:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495913600:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760777.830965:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26578944, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd009 00010000:00010000:2.0:1789760777.830974:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454793b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.830990:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760777.830994:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.831001:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454793b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.831014:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454793b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760777.863021:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26578944 -> 26583040 00010000:00010000:2.0:1789760777.863040:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454793b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.863053:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454793b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.863061:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454793b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.863068:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760777.863072:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454793b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.863082:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454793b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.863099:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454793b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.863107:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454793b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.863218:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495913600:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32295us (32643us total) trans 4294973806 rc 0/0 00000100:00100000:2.0:1789760777.863227:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6518 00000100:00100000:2.0:1789760777.863237:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762091008422 time 762151621785 state 0x2 ref 0 00000001:00080000:2.0:1789760777.863275:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973806 is committed 00000100:00100000:2.0:1789760777.868973:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760777.869572:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495913728 00000100:00100000:0.0:1789760777.869587:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495913728 00000100:00100000:0.0:1789760777.869641:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762091008422 time 762151621785 state 0x0 ref 1 00000100:00100000:0.0:1789760777.869656:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762151882201 time 762151621785 state 0x1 ref 1 00000100:00100000:0.0:1789760777.869687:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762151882201 time 762190403865 state 0x0 ref 1 00000100:00100000:0.0:1789760777.869703:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495913728:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760777.869757:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26583040, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd009 00010000:00010000:0.0:1789760777.869766:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547942 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.869791:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760777.869795:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.869802:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547942 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.869815:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547942 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.914302:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973807 is committed 00002000:00010000:0.0:1789760777.914459:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26583040 -> 26587136 00010000:00010000:0.0:1789760777.914475:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547942 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.914489:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547942 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.914498:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547942 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.914505:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760777.914510:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547942 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.914518:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547942 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.914537:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547942 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.914545:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547942 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.914639:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495913728:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44933us (45671us total) trans 4294973807 rc 0/0 00000100:00100000:0.0:1789760777.914652:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6519 00000100:00100000:0.0:1789760777.914662:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762151882201 time 762190403865 state 0x2 ref 0 00000100:00100000:3.0:1789760777.920641:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760777.920716:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495913856 00000100:00100000:0.0:1789760777.920726:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495913856 00000100:00100000:0.0:1789760777.920858:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762151882201 time 762190403865 state 0x0 ref 1 00000100:00100000:0.0:1789760777.920873:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762190664281 time 762190403865 state 0x1 ref 1 00000100:00100000:0.0:1789760777.920908:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762190664281 time 762241625271 state 0x0 ref 1 00000100:00100000:0.0:1789760777.920921:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495913856:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760777.920967:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26587136, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd009 00010000:00010000:0.0:1789760777.920975:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547949 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.920990:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760777.920994:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.920999:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547949 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.921011:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547949 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.965041:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973808 is committed 00002000:00010000:0.0:1789760777.965493:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26587136 -> 26591232 00010000:00010000:0.0:1789760777.965508:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547949 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.965522:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547949 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.965531:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547949 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.965538:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760777.965542:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547949 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.965551:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547949 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.965570:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547949 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.965577:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547949 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.965716:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495913856:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44792us (45078us total) trans 4294973808 rc 0/0 00000100:00100000:0.0:1789760777.965738:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6520 00000100:00100000:0.0:1789760777.965748:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762190664281 time 762241625271 state 0x2 ref 0 00000100:00100000:1.0:1789760777.969796:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760777.969942:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495913984 00000100:00100000:2.0:1789760777.969951:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495913984 00000100:00100000:2.0:1789760777.970004:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762190664281 time 762241625271 state 0x0 ref 1 00000100:00100000:2.0:1789760777.970019:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762241885687 time 762241625271 state 0x1 ref 1 00000100:00100000:2.0:1789760777.970102:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762241885687 time 762290809003 state 0x0 ref 1 00000100:00100000:2.0:1789760777.970116:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495913984:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760777.970164:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26591232, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd009 00010000:00010000:2.0:1789760777.970173:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547950 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.970188:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760777.970193:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.970200:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547950 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.970213:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547950 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.997378:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973809 is committed 00002000:00010000:2.0:1789760777.997644:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26591232 -> 26595328 00010000:00010000:2.0:1789760777.997661:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547950 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.997674:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547950 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.997683:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547950 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.997690:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760777.997695:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547950 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.997704:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547950 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.997723:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547950 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.997731:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547950 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.997897:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495913984:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27778us (28105us total) trans 4294973809 rc 0/0 00000100:00100000:2.0:1789760777.997907:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6521 00000100:00100000:2.0:1789760777.997917:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762241885687 time 762290809003 state 0x2 ref 0 00000100:00100000:2.0:1789760778.002714:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760778.003267:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495914112 00000100:00100000:0.0:1789760778.003299:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495914112 00000100:00100000:0.0:1789760778.003364:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762241885687 time 762290809003 state 0x0 ref 1 00000100:00100000:0.0:1789760778.003379:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762291069419 time 762290809003 state 0x1 ref 1 00000100:00100000:0.0:1789760778.003405:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762291069419 time 762324122070 state 0x0 ref 1 00000100:00100000:0.0:1789760778.003418:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03096680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495914112:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760778.003644:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26595328, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd009 00010000:00010000:0.0:1789760778.003654:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547957 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.003670:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760778.003674:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.003680:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547957 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.003694:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547957 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.034214:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973810 is committed 00002000:00010000:0.0:1789760778.034549:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26595328 -> 26599424 00010000:00010000:0.0:1789760778.034566:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547957 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.034580:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547957 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.034596:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547957 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.034604:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760778.034610:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547957 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.034619:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547957 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.034640:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547957 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.034648:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547957 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.034799:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03096680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495914112:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31379us (32091us total) trans 4294973810 rc 0/0 00000100:00100000:0.0:1789760778.034810:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6522 00000100:00100000:0.0:1789760778.034821:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762291069419 time 762324122070 state 0x2 ref 0 00000100:00100000:1.0:1789760778.039049:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760778.039174:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495914240 00000100:00100000:0.0:1789760778.039184:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495914240 00000100:00100000:0.0:1789760778.039237:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762291069419 time 762324122070 state 0x0 ref 1 00000100:00100000:0.0:1789760778.039254:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762324382486 time 762324122070 state 0x1 ref 1 00000100:00100000:0.0:1789760778.039348:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762324382486 time 762360063616 state 0x0 ref 1 00000100:00100000:0.0:1789760778.039362:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036398450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495914240:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760778.039410:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26599424, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd009 00010000:00010000:0.0:1789760778.039418:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454795e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.039445:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760778.039449:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.039455:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454795e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.039469:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454795e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.074065:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973811 is committed 00002000:00010000:0.0:1789760778.074229:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26599424 -> 26603520 00010000:00010000:0.0:1789760778.074245:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454795e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.074258:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454795e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.074268:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454795e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.074275:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760778.074309:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454795e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.074319:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454795e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.074404:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454795e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.074413:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454795e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.074568:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036398450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495914240:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35205us (35524us total) trans 4294973811 rc 0/0 00000100:00100000:0.0:1789760778.074579:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6523 00000100:00100000:0.0:1789760778.074590:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762324382486 time 762360063616 state 0x2 ref 0 00000100:00100000:1.0:1789760778.079864:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00010000:00010000:0.0:1789760778.080027:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789760778.080061:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495914368 00000100:00100000:2.0:1789760778.080070:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495914368 00000100:00100000:2.0:1789760778.080188:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762324382486 time 762360063616 state 0x0 ref 1 00000100:00100000:2.0:1789760778.080204:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762360324032 time 762360063616 state 0x1 ref 1 00000100:00100000:2.0:1789760778.080274:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762360324032 time 762400989860 state 0x0 ref 1 00000100:00100000:2.0:1789760778.080313:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495914368:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760778.080511:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26603520, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd009 00010000:00010000:2.0:1789760778.080521:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547965 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.080542:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760778.080546:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.080553:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547965 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.080566:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547965 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.110708:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973812 is committed 00002000:00010000:2.0:1789760778.110855:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26603520 -> 26607616 00010000:00010000:2.0:1789760778.110871:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547965 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.110884:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547965 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.110893:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547965 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.110900:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760778.110905:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547965 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.110914:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547965 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.110934:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547965 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.110943:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547965 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.111036:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495914368:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30721us (31176us total) trans 4294973812 rc 0/0 00000100:00100000:2.0:1789760778.111046:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6524 00000100:00100000:2.0:1789760778.111056:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762360324032 time 762400989860 state 0x2 ref 0 00000100:00100000:2.0:1789760778.116638:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760778.116715:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495914496 00000100:00100000:0.0:1789760778.116724:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495914496 00000100:00100000:0.0:1789760778.116785:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762360324032 time 762400989860 state 0x0 ref 1 00000100:00100000:0.0:1789760778.116800:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762401250276 time 762400989860 state 0x1 ref 1 00000100:00100000:0.0:1789760778.116832:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762401250276 time 762437548600 state 0x0 ref 1 00000100:00100000:0.0:1789760778.116845:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495914496:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760778.116895:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26607616, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd009 00010000:00010000:0.0:1789760778.116904:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454796c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.116920:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760778.116925:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.116931:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454796c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.116944:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454796c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760778.164538:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26607616 -> 26611712 00010000:00010000:0.0:1789760778.164559:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454796c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.164572:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454796c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.164582:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454796c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.164590:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760778.164595:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454796c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.164605:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454796c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.164625:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454796c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.164633:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454796c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.164764:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495914496:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47917us (48131us total) trans 4294973813 rc 0/0 00000100:00100000:0.0:1789760778.164784:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6525 00000100:00100000:0.0:1789760778.164795:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762401250276 time 762437548600 state 0x2 ref 0 00000001:00080000:3.0:1789760778.167056:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973813 is committed 00000100:00100000:3.0:1789760778.167901:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760778.168208:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495914624 00000100:00100000:0.0:1789760778.168217:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495914624 00000100:00100000:0.0:1789760778.168271:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762401250276 time 762437548600 state 0x0 ref 1 00000100:00100000:0.0:1789760778.168349:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762437809016 time 762437548600 state 0x1 ref 1 00000100:00100000:0.0:1789760778.168413:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762437809016 time 762489128646 state 0x0 ref 1 00000100:00100000:0.0:1789760778.168426:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495914624:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760778.168473:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26611712, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd009 00010000:00010000:0.0:1789760778.168482:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547973 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.168497:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760778.168502:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.168509:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547973 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.168523:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547973 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.187667:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973814 is committed 00002000:00010000:1.0:1789760778.187777:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26611712 -> 26615808 00010000:00010000:1.0:1789760778.187793:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547973 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.187804:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547973 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.187812:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547973 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.187819:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760778.187823:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547973 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.187830:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547973 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.187848:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547973 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.187854:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547973 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.187942:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495914624:12345-192.168.203.9@tcp:4:dd.0 Request processed in 19513us (20041us total) trans 4294973814 rc 0/0 00000100:00100000:1.0:1789760778.187953:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6526 00000100:00100000:1.0:1789760778.187962:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762437809016 time 762489128646 state 0x2 ref 0 00000100:00100000:1.0:1789760778.192355:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760778.192581:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495914752 00000100:00100000:2.0:1789760778.192592:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495914752 00000100:00100000:2.0:1789760778.192650:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762437809016 time 762489128646 state 0x0 ref 1 00000100:00100000:2.0:1789760778.192666:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762489389062 time 762489128646 state 0x1 ref 1 00000100:00100000:2.0:1789760778.192693:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762489389062 time 762513409306 state 0x0 ref 1 00000100:00100000:2.0:1789760778.192705:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495914752:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760778.192944:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26615808, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd009 00010000:00010000:2.0:1789760778.192954:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454797a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.192971:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760778.192976:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.192983:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454797a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.192997:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454797a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.214888:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973815 is committed 00002000:00010000:1.0:1789760778.214992:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26615808 -> 26619904 00010000:00010000:1.0:1789760778.215008:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454797a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.215021:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454797a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.215031:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454797a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.215039:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760778.215044:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454797a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.215053:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454797a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.215072:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454797a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.215081:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454797a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.215174:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495914752:12345-192.168.203.9@tcp:4:dd.0 Request processed in 22466us (22823us total) trans 4294973815 rc 0/0 00000100:00100000:1.0:1789760778.215185:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6527 00000100:00100000:1.0:1789760778.215194:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762489389062 time 762513409306 state 0x2 ref 0 00000100:00100000:2.0:1789760778.223208:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760778.223320:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495914880 00000100:00100000:0.0:1789760778.223330:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495914880 00000100:00100000:0.0:1789760778.223384:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762489389062 time 762513409306 state 0x0 ref 1 00000100:00100000:0.0:1789760778.223410:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762513669722 time 762513409306 state 0x1 ref 1 00000100:00100000:0.0:1789760778.223442:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762513669722 time 762544157414 state 0x0 ref 1 00000100:00100000:0.0:1789760778.223455:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03096680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495914880:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760778.223502:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26619904, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd009 00010000:00010000:0.0:1789760778.223512:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547981 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.223529:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760778.223533:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.223539:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547981 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.223553:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547981 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.261559:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973816 is committed 00002000:00010000:3.0:1789760778.261682:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26619904 -> 26624000 00010000:00010000:3.0:1789760778.261697:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547981 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.261710:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547981 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.261719:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547981 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.261727:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760778.261732:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547981 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.261741:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547981 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.261760:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547981 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.261778:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547981 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.261874:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03096680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495914880:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38417us (38673us total) trans 4294973816 rc 0/0 00000100:00100000:3.0:1789760778.261885:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6528 00000100:00100000:3.0:1789760778.261895:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762513669722 time 762544157414 state 0x2 ref 0 00000100:00100000:3.0:1789760778.269524:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760778.269904:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495915008 00000100:00100000:1.0:1789760778.269915:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495915008 00000100:00100000:1.0:1789760778.269972:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762513669722 time 762544157414 state 0x0 ref 1 00000100:00100000:1.0:1789760778.269988:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762544417830 time 762544157414 state 0x1 ref 1 00000100:00100000:1.0:1789760778.270062:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762544417830 time 762590778761 state 0x0 ref 1 00000100:00100000:1.0:1789760778.270076:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495915008:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760778.270139:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26624000, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd009 00010000:00010000:1.0:1789760778.270148:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354547988 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.270164:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760778.270168:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.270174:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354547988 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.270187:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354547988 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760778.304407:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26624000 -> 26628096 00010000:00010000:0.0:1789760778.304429:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354547988 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.304443:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354547988 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.304453:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354547988 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.304461:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760778.304467:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354547988 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.304477:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354547988 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.304497:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354547988 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.304506:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354547988 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.304605:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495915008:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34526us (35084us total) trans 4294973817 rc 0/0 00000100:00100000:0.0:1789760778.304619:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6529 00000100:00100000:0.0:1789760778.304628:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762544417830 time 762590778761 state 0x2 ref 0 00000001:00080000:3.0:1789760778.305144:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973817 is committed 00000100:00100000:0.0:1789760778.308739:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760778.308950:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495915136 00000100:00100000:1.0:1789760778.308962:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495915136 00000100:00100000:1.0:1789760778.309020:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762544417830 time 762590778761 state 0x0 ref 1 00000100:00100000:1.0:1789760778.309035:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762591039177 time 762590778761 state 0x1 ref 1 00000100:00100000:1.0:1789760778.309071:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762591039177 time 762629787925 state 0x0 ref 1 00000100:00100000:1.0:1789760778.309083:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495915136:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760778.309133:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26628096, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd009 00010000:00010000:1.0:1789760778.309143:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454798f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.309159:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760778.309164:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.309171:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454798f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.309185:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454798f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760778.358103:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26628096 -> 26632192 00010000:00010000:1.0:1789760778.358123:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454798f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.358137:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454798f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.358146:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454798f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.358154:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760778.358160:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454798f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.358168:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454798f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.358190:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454798f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.358196:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454798f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.358322:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495915136:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49237us (49589us total) trans 4294973818 rc 0/0 00000100:00100000:1.0:1789760778.358332:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6530 00000100:00100000:1.0:1789760778.358342:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762591039177 time 762629787925 state 0x2 ref 0 00000001:00080000:3.0:1789760778.358595:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973818 is committed 00000100:00100000:1.0:1789760778.362261:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760778.362343:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495915264 00000100:00100000:3.0:1789760778.362355:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495915264 00000100:00100000:2.0:1789760778.362440:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762591039177 time 762629787925 state 0x0 ref 1 00000100:00100000:2.0:1789760778.362456:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762630048341 time 762629787925 state 0x1 ref 1 00000100:00100000:2.0:1789760778.362486:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762630048341 time 762683203023 state 0x0 ref 1 00000100:00100000:2.0:1789760778.362499:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495915264:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760778.362548:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26632192, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd009 00010000:00010000:2.0:1789760778.362558:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547996 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.362572:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760778.362576:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.362583:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547996 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.362596:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547996 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.397947:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973819 is committed 00002000:00010000:3.0:1789760778.398038:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26632192 -> 26636288 00010000:00010000:3.0:1789760778.398054:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547996 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.398067:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547996 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.398218:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547996 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.398226:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760778.398230:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547996 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.398239:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547996 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.398258:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547996 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.398266:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547996 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.398557:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495915264:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36054us (36297us total) trans 4294973819 rc 0/0 00000100:00100000:3.0:1789760778.398568:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6531 00000100:00100000:3.0:1789760778.398578:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762630048341 time 762683203023 state 0x2 ref 0 00000100:00100000:0.0:1789760778.406570:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760778.406647:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495915392 00000100:00100000:1.0:1789760778.406657:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495915392 00000100:00100000:1.0:1789760778.406710:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762630048341 time 762683203023 state 0x0 ref 1 00000100:00100000:1.0:1789760778.406725:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762683463439 time 762683203023 state 0x1 ref 1 00000100:00100000:1.0:1789760778.406761:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762683463439 time 762727477785 state 0x0 ref 1 00000100:00100000:1.0:1789760778.406783:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495915392:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760778.406828:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26636288, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd009 00010000:00010000:1.0:1789760778.406837:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454799d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.406852:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760778.406857:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.406963:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454799d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.406978:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454799d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760778.443248:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26636288 -> 26640384 00010000:00010000:3.0:1789760778.443270:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454799d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.443315:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454799d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.443324:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454799d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.443332:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760778.443336:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454799d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.443345:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454799d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.443365:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454799d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.443373:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454799d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.443465:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495915392:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36680us (36899us total) trans 4294973820 rc 0/0 00000100:00100000:3.0:1789760778.443479:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6532 00000100:00100000:3.0:1789760778.443489:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762683463439 time 762727477785 state 0x2 ref 0 00000001:00080000:0.0:1789760778.443605:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973820 is committed 00000100:00100000:0.0:1789760778.451251:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760778.451657:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495915520 00000100:00100000:1.0:1789760778.451668:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495915520 00000100:00100000:1.0:1789760778.451724:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762683463439 time 762727477785 state 0x0 ref 1 00000100:00100000:1.0:1789760778.451739:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762727738201 time 762727477785 state 0x1 ref 1 00000100:00100000:1.0:1789760778.452295:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762727738201 time 762772992413 state 0x0 ref 1 00000100:00100000:1.0:1789760778.452312:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495915520:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760778.452358:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26640384, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd009 00010000:00010000:1.0:1789760778.452367:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545479a4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.452388:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760778.452393:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.452399:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545479a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.452412:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545479a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.496185:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973821 is committed 00002000:00010000:1.0:1789760778.496365:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26640384 -> 26644480 00010000:00010000:1.0:1789760778.496380:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545479a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.496393:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545479a4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.496401:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545479a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.496408:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760778.496413:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545479a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.496421:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545479a4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.496441:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545479a4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.496448:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545479a4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.496549:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495915520:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44236us (45301us total) trans 4294973821 rc 0/0 00000100:00100000:1.0:1789760778.496560:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6533 00000100:00100000:1.0:1789760778.496571:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762727738201 time 762772992413 state 0x2 ref 0 00000100:00100000:2.0:1789760778.499624:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760778.499724:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495915648 00000100:00100000:3.0:1789760778.499734:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495915648 00000100:00100000:3.0:1789760778.500166:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762727738201 time 762772992413 state 0x0 ref 1 00000100:00100000:3.0:1789760778.500183:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762773252829 time 762772992413 state 0x1 ref 1 00000100:00100000:3.0:1789760778.500213:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762773252829 time 762820930219 state 0x0 ref 1 00000100:00100000:3.0:1789760778.500226:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495915648:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760778.500274:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26644480, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd009 00010000:00010000:3.0:1789760778.500317:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479ab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.500344:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760778.500348:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.500354:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.500368:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.531390:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff929036f66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:59c4e264-9b75-482b-8813-3d1dc124f384:3317:1876699520314240:0@lo:400:kworker.0 00000100:00100000:3.0:1789760778.531545:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760778.531608:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f0492d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3317:1876699520314752:0@lo:400:kworker.0 00000100:00100000:3.0:1789760778.531637:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760778.531664:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1789760778.531669:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff929036f64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3317:1876699520314368:0@lo:400:kworker.0 00000100:00100000:3.0:1789760778.531691:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760778.531708:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f04293b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3317:1876699520314880:0@lo:400:kworker.0 00000100:00100000:3.0:1789760778.531731:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760778.531774:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff929036f65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520314496:0@lo:400:kworker.0 00000100:00100000:3.0:1789760778.531798:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760778.531816:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789760778.531821:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f0492d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520314624:0@lo:400:kworker.0 00000100:00100000:3.0:1789760778.531847:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760778.531850:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520314240 00000100:00100000:0.0:1789760778.531859:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520314240 00000100:00100000:3.0:1789760778.531879:0:8003:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520314752 00000100:00100000:3.0:1789760778.531884:0:8003:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520314752 00000100:00100000:0.0:1789760778.531909:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 415 00000100:00100000:0.0:1789760778.531921:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290122cd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:59c4e264-9b75-482b-8813-3d1dc124f384+19:3317:x1876699520314240:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760778.531930:0:8003:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520314368 00000100:00100000:3.0:1789760778.531934:0:8003:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520314368 00000100:00100000:3.0:1789760778.531954:0:8003:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520314496 00000100:00100000:3.0:1789760778.531957:0:8003:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520314496 00000100:00100000:3.0:1789760778.531977:0:8003:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 517 00000100:00100000:0.0:1789760778.531983:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290122cd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:59c4e264-9b75-482b-8813-3d1dc124f384+19:3317:x1876699520314240:12345-0@lo:400:kworker.0 Request processed in 64us (437us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760778.531987:0:8003:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1876699520314752:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760778.531996:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 415 00000100:00100000:3.0:1789760778.532031:0:8003:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1876699520314752:12345-0@lo:400:kworker.0 Request processed in 45us (392us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760778.532039:0:8003:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 517 00000100:00100000:3.0:1789760778.532059:0:8003:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 518 00000100:00100000:0.0:1789760778.532068:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff929036f66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:59c4e264-9b75-482b-8813-3d1dc124f384:3317:1876699520314240:0@lo:400:kworker.0 00000100:00100000:3.0:1789760778.532069:0:8003:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04291500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1876699520314368:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760778.532091:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f0492d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3317:1876699520314752:0@lo:400:kworker.0 00000100:00100000:3.0:1789760778.532112:0:8003:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04291500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1876699520314368:12345-0@lo:400:kworker.0 Request processed in 46us (418us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760778.532119:0:8003:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 518 00000100:00100000:3.0:1789760778.532132:0:8003:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 519 00000100:00100000:3.0:1789760778.532138:0:8003:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04292a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520314496:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760778.532168:0:8003:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04292a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520314496:12345-0@lo:400:kworker.0 Request processed in 31us (369us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760778.532174:0:8003:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 519 00000100:00100000:3.0:1789760778.532195:0:13058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520314880 00000100:00100000:3.0:1789760778.532199:0:13058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520314880 00000100:00100000:3.0:1789760778.532225:0:13058:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520314624 00000100:00100000:3.0:1789760778.532228:0:13058:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520314624 00000100:00100000:3.0:1789760778.532249:0:13058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 238 00000100:00100000:3.0:1789760778.532256:0:13058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04291180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3317:x1876699520314880:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760778.532309:0:13058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04291180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3317:x1876699520314880:12345-0@lo:400:kworker.0 Request processed in 54us (577us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760778.532315:0:13058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 238 00000100:00100000:3.0:1789760778.532326:0:13058:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 239 00000100:00100000:3.0:1789760778.532333:0:13058:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520314624:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760778.532359:0:13058:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520314624:12345-0@lo:400:kworker.0 Request processed in 28us (513us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760778.532366:0:13058:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 239 00000100:00080000:3.0:1789760778.532389:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789760778.532393:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789760778.532395:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760778.532425:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f0492d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520314624:0@lo:400:kworker.0 00000100:00100000:3.0:1789760778.532438:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff929036f65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520314496:0@lo:400:kworker.0 00000100:00100000:0.0:1789760778.533800:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff929036f64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3317:1876699520314368:0@lo:400:kworker.0 00000100:00100000:0.0:1789760778.533816:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f04293b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3317:1876699520314880:0@lo:400:kworker.0 00000001:00080000:0.0:1789760778.564519:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973822 is committed 00002000:00010000:3.0:1789760778.564630:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26644480 -> 26648576 00010000:00010000:3.0:1789760778.564645:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.564659:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479ab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.564670:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.564678:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760778.564683:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.564693:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.564715:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479ab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.564723:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479ab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.564837:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495915648:12345-192.168.203.9@tcp:4:dd.0 Request processed in 64608us (65217us total) trans 4294973822 rc 0/0 00000100:00100000:3.0:1789760778.564852:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6534 00000100:00100000:3.0:1789760778.564861:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762773252829 time 762820930219 state 0x2 ref 0 00000100:00100000:3.0:1789760778.571738:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760778.571848:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495915776 00000100:00100000:1.0:1789760778.571858:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495915776 00000100:00100000:1.0:1789760778.571909:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762773252829 time 762820930219 state 0x0 ref 1 00000100:00100000:1.0:1789760778.571925:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762821190635 time 762820930219 state 0x1 ref 1 00000100:00100000:1.0:1789760778.571989:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762821190635 time 762892705595 state 0x0 ref 1 00000100:00100000:1.0:1789760778.572002:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495915776:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760778.572052:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26648576, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd009 00010000:00010000:1.0:1789760778.572062:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545479b2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.572079:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760778.572084:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.572091:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545479b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.572110:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545479b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.615495:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973823 is committed 00002000:00010000:3.0:1789760778.616516:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26648576 -> 26652672 00010000:00010000:3.0:1789760778.616535:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545479b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.616549:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545479b2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.616558:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545479b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.616566:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760778.616572:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545479b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.616581:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545479b2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.616600:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545479b2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.616607:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545479b2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.616760:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495915776:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44754us (45025us total) trans 4294973823 rc 0/0 00000100:00100000:3.0:1789760778.616779:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6535 00000100:00100000:3.0:1789760778.616789:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762821190635 time 762892705595 state 0x2 ref 0 00000100:00100000:0.0:1789760778.623229:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760778.623420:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495915904 00000100:00100000:1.0:1789760778.623430:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495915904 00000100:00100000:1.0:1789760778.623484:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762821190635 time 762892705595 state 0x0 ref 1 00000100:00100000:1.0:1789760778.623496:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762892966011 time 762892705595 state 0x1 ref 1 00000100:00100000:1.0:1789760778.623524:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762892966011 time 762944241866 state 0x0 ref 1 00000100:00100000:1.0:1789760778.623533:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495915904:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760778.623570:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26652672, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd009 00010000:00010000:1.0:1789760778.623577:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545479b9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.623590:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760778.623593:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.623597:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545479b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.623608:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545479b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.654994:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973824 is committed 00002000:00010000:1.0:1789760778.655095:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26652672 -> 26656768 00002000:00010000:1.0:1789760778.655101:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760777 -> 1789760778 00002000:00010000:1.0:1789760778.655104:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760777 -> 1789760778 00010000:00010000:1.0:1789760778.655117:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545479b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.655129:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545479b9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.655138:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545479b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.655145:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760778.655150:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545479b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.655159:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545479b9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.655179:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545479b9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.655187:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545479b9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.655449:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495915904:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31741us (32055us total) trans 4294973824 rc 0/0 00000100:00100000:1.0:1789760778.655462:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6536 00000100:00100000:1.0:1789760778.655472:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762892966011 time 762944241866 state 0x2 ref 0 00000100:00100000:2.0:1789760778.670705:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760778.670796:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495916032 00000100:00100000:3.0:1789760778.670806:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495916032 00000100:00100000:3.0:1789760778.670860:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762892966011 time 762944241866 state 0x0 ref 1 00000100:00100000:3.0:1789760778.670875:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762944502282 time 762944241866 state 0x1 ref 1 00000100:00100000:3.0:1789760778.670908:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762944502282 time 762991624569 state 0x0 ref 1 00000100:00100000:3.0:1789760778.670920:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495916032:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760778.670966:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26656768, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd009 00010000:00010000:3.0:1789760778.670976:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479c0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.670992:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760778.670997:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.671003:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.671020:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760778.706005:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26656768 -> 26660864 00010000:00010000:3.0:1789760778.706024:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.706039:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479c0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.706049:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.706057:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760778.706063:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.706073:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479c0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.706096:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479c0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.706104:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479c0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.706195:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495916032:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35272us (35494us total) trans 4294973825 rc 0/0 00000100:00100000:3.0:1789760778.706210:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6537 00000100:00100000:3.0:1789760778.706220:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762944502282 time 762991624569 state 0x2 ref 0 00000001:00080000:3.0:1789760778.706261:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973825 is committed 00000100:00100000:3.0:1789760778.715038:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760778.715094:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495916160 00000100:00100000:1.0:1789760778.715105:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495916160 00000100:00100000:1.0:1789760778.715160:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762944502282 time 762991624569 state 0x0 ref 1 00000100:00100000:1.0:1789760778.715176:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762991884985 time 762991624569 state 0x1 ref 1 00000100:00100000:1.0:1789760778.715210:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762991884985 time 763035926329 state 0x0 ref 1 00000100:00100000:1.0:1789760778.715221:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495916160:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760778.715268:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26660864, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd009 00010000:00010000:1.0:1789760778.715316:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545479c7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.715347:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760778.715352:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.715359:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545479c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.715375:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545479c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.743107:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973826 is committed 00002000:00010000:1.0:1789760778.743540:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26660864 -> 26664960 00010000:00010000:1.0:1789760778.743555:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545479c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.743574:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545479c7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.743584:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545479c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.743591:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760778.743596:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545479c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.743605:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545479c7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.743624:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545479c7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.743632:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545479c7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.743817:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495916160:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28592us (28787us total) trans 4294973826 rc 0/0 00000100:00100000:1.0:1789760778.743828:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6538 00000100:00100000:1.0:1789760778.743838:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762991884985 time 763035926329 state 0x2 ref 0 00000100:00100000:1.0:1789760778.749114:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760778.749480:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495916288 00000100:00100000:2.0:1789760778.749490:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495916288 00000100:00100000:0.0:1789760778.749611:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 762991884985 time 763035926329 state 0x0 ref 1 00000100:00100000:0.0:1789760778.749627:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763036186745 time 763035926329 state 0x1 ref 1 00000100:00100000:0.0:1789760778.749655:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763036186745 time 763070372658 state 0x0 ref 1 00000100:00100000:0.0:1789760778.749668:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495916288:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760778.749718:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26664960, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd009 00010000:00010000:0.0:1789760778.749726:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545479ce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.749743:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760778.749748:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.749754:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545479ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.749776:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545479ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.788526:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973827 is committed 00002000:00010000:1.0:1789760778.788975:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26664960 -> 26669056 00010000:00010000:1.0:1789760778.788991:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545479ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.789004:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545479ce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.789013:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545479ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.789021:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760778.789026:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545479ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.789035:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545479ce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.789053:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545479ce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.789061:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545479ce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.789207:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495916288:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39537us (40098us total) trans 4294973827 rc 0/0 00000100:00100000:1.0:1789760778.789218:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6539 00000100:00100000:1.0:1789760778.789228:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763036186745 time 763070372658 state 0x2 ref 0 00000100:00100000:2.0:1789760778.795218:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760778.795415:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495916416 00000100:00100000:3.0:1789760778.795426:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495916416 00000100:00100000:3.0:1789760778.795602:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763036186745 time 763070372658 state 0x0 ref 1 00000100:00100000:3.0:1789760778.795620:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763070633074 time 763070372658 state 0x1 ref 1 00000100:00100000:3.0:1789760778.795651:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763070633074 time 763116367850 state 0x0 ref 1 00000100:00100000:3.0:1789760778.795666:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495916416:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760778.795722:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26669056, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd009 00010000:00010000:3.0:1789760778.795843:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545479d5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.795861:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760778.795866:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.795873:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545479d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.795887:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545479d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.836594:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973828 is committed 00002000:00010000:1.0:1789760778.836706:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26669056 -> 26673152 00010000:00010000:1.0:1789760778.836721:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545479d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.836733:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545479d5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.836754:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545479d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.836769:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760778.836774:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545479d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.836784:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545479d5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.836803:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545479d5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.836811:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545479d5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.836969:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495916416:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41301us (41755us total) trans 4294973828 rc 0/0 00000100:00100000:1.0:1789760778.836980:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6540 00000100:00100000:1.0:1789760778.836991:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763070633074 time 763116367850 state 0x2 ref 0 00000100:00100000:3.0:1789760778.843344:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760778.843450:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495916544 00000100:00100000:1.0:1789760778.843462:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495916544 00000100:00100000:1.0:1789760778.843516:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763070633074 time 763116367850 state 0x0 ref 1 00000100:00100000:1.0:1789760778.843532:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763116628266 time 763116367850 state 0x1 ref 1 00000100:00100000:1.0:1789760778.843560:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763116628266 time 763164276777 state 0x0 ref 1 00000100:00100000:1.0:1789760778.843573:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495916544:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760778.843616:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26673152, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd009 00010000:00010000:1.0:1789760778.843627:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545479dc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.843644:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760778.843648:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.843656:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545479dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.843669:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545479dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.877650:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973829 is committed 00002000:00010000:3.0:1789760778.877845:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26673152 -> 26677248 00010000:00010000:3.0:1789760778.877861:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545479dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.877874:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545479dc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.877884:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545479dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.877892:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760778.877896:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545479dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.877905:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545479dc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.877925:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545479dc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.877932:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545479dc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.878029:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495916544:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34453us (34698us total) trans 4294973829 rc 0/0 00000100:00100000:3.0:1789760778.878040:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6541 00000100:00100000:3.0:1789760778.878050:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763116628266 time 763164276777 state 0x2 ref 0 00000100:00100000:0.0:1789760778.889440:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760778.890257:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495916672 00000100:00100000:1.0:1789760778.890268:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495916672 00000100:00100000:1.0:1789760778.890368:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763116628266 time 763164276777 state 0x0 ref 1 00000100:00100000:1.0:1789760778.890388:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763164537193 time 763164276777 state 0x1 ref 1 00000100:00100000:1.0:1789760778.890423:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763164537193 time 763211140083 state 0x0 ref 1 00000100:00100000:1.0:1789760778.890436:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495916672:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760778.890489:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26677248, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd009 00010000:00010000:1.0:1789760778.890498:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545479e3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.890515:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760778.890519:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.890526:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545479e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.890540:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545479e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.920621:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973830 is committed 00002000:00010000:3.0:1789760778.920732:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26677248 -> 26681344 00010000:00010000:3.0:1789760778.920758:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545479e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.920780:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545479e3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.920789:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545479e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.920797:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760778.920802:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545479e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.920811:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545479e3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.920831:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545479e3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.920838:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545479e3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.920947:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495916672:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30509us (31510us total) trans 4294973830 rc 0/0 00000100:00100000:3.0:1789760778.920958:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6542 00000100:00100000:3.0:1789760778.920968:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763164537193 time 763211140083 state 0x2 ref 0 00000100:00100000:2.0:1789760778.929813:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760778.929965:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495916800 00000100:00100000:1.0:1789760778.929975:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495916800 00000100:00100000:1.0:1789760778.930027:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763164537193 time 763211140083 state 0x0 ref 1 00000100:00100000:1.0:1789760778.930042:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763211400499 time 763211140083 state 0x1 ref 1 00000100:00100000:1.0:1789760778.930071:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763211400499 time 763250787788 state 0x0 ref 1 00000100:00100000:1.0:1789760778.930083:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495916800:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760778.930130:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26681344, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd009 00010000:00010000:1.0:1789760778.930138:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545479ea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.930155:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760778.930160:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.930166:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545479ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.930179:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545479ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.958004:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973831 is committed 00002000:00010000:3.0:1789760778.958092:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26681344 -> 26685440 00010000:00010000:3.0:1789760778.958108:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545479ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.958120:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545479ea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.958129:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545479ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.958136:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760778.958140:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545479ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.958149:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545479ea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.958170:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545479ea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.958183:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x49367483545479ea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.958397:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495916800:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28312us (28591us total) trans 4294973831 rc 0/0 00000100:00100000:3.0:1789760778.958408:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6543 00000100:00100000:3.0:1789760778.958418:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763211400499 time 763250787788 state 0x2 ref 0 00000100:00100000:0.0:1789760778.962107:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760778.962178:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495916928 00000100:00100000:3.0:1789760778.962188:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495916928 00000100:00100000:3.0:1789760778.962250:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763211400499 time 763250787788 state 0x0 ref 1 00000100:00100000:3.0:1789760778.962265:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763251048204 time 763250787788 state 0x1 ref 1 00000100:00100000:3.0:1789760778.962325:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763251048204 time 763283041532 state 0x0 ref 1 00000100:00100000:3.0:1789760778.962338:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495916928:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760778.962385:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26685440, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd009 00010000:00010000:3.0:1789760778.962395:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479f1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.962410:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760778.962413:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.962419:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.962435:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.985369:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973832 is committed 00002000:00010000:1.0:1789760778.986495:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26685440 -> 26689536 00010000:00010000:1.0:1789760778.986514:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.986527:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479f1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.986536:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.986554:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760778.986560:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.986569:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479f1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.986719:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479f1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.986730:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545479f1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.986840:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495916928:12345-192.168.203.9@tcp:4:dd.0 Request processed in 24498us (24736us total) trans 4294973832 rc 0/0 00000100:00100000:1.0:1789760778.986851:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6544 00000100:00100000:1.0:1789760778.986861:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763251048204 time 763283041532 state 0x2 ref 0 00000100:00100000:0.0:1789760778.990973:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760778.991081:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495917056 00000100:00100000:3.0:1789760778.991092:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495917056 00000100:00100000:3.0:1789760778.991145:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763251048204 time 763283041532 state 0x0 ref 1 00000100:00100000:3.0:1789760778.991162:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763283301948 time 763283041532 state 0x1 ref 1 00000100:00100000:3.0:1789760778.991241:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763283301948 time 763311957684 state 0x0 ref 1 00000100:00100000:3.0:1789760778.991254:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495917056:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760778.991334:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26689536, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd009 00010000:00010000:3.0:1789760778.991343:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545479f8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.991468:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760778.991473:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.991479:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545479f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.991494:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545479f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.020010:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973833 is committed 00002000:00010000:2.0:1789760779.020164:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26689536 -> 26693632 00010000:00010000:2.0:1789760779.020180:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545479f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.020194:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545479f8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.020204:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545479f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.020212:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760779.020217:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545479f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.020226:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545479f8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.020246:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545479f8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.020258:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545479f8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.020470:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495917056:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29213us (29505us total) trans 4294973833 rc 0/0 00000100:00100000:2.0:1789760779.020481:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6545 00000100:00100000:2.0:1789760779.020490:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763283301948 time 763311957684 state 0x2 ref 0 00000100:00100000:2.0:1789760779.024734:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760779.024917:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495917184 00000100:00100000:3.0:1789760779.024926:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495917184 00000100:00100000:3.0:1789760779.024993:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763283301948 time 763311957684 state 0x0 ref 1 00000100:00100000:3.0:1789760779.025008:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763312218100 time 763311957684 state 0x1 ref 1 00000100:00100000:3.0:1789760779.025035:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763312218100 time 763345751839 state 0x0 ref 1 00000100:00100000:3.0:1789760779.025048:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495917184:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760779.025201:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26693632, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd009 00010000:00010000:3.0:1789760779.025211:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545479ff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.025226:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760779.025230:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.025236:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545479ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.025249:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545479ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760779.025707:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760779.038911:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789760779.038937:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f033b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520315008:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760779.039008:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760779.039032:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520315008 00000100:00100000:3.0:1789760779.039040:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520315008 00000100:00100000:3.0:1789760779.039099:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 214 00000100:00100000:3.0:1789760779.039110:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520315008:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760779.039141:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760779.039185:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520315008:12345-0@lo:13:osp-pre-1-0.0 Request processed in 78us (177us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760779.039195:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 214 00000100:00100000:1.0:1789760779.039928:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f033b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520315008:0@lo:13:osp-pre-1-0.0 00002000:00010000:1.0:1789760779.067722:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26693632 -> 26697728 00010000:00010000:1.0:1789760779.067751:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545479ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.067775:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545479ff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.067785:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545479ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.067793:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760779.067798:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545479ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.067807:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545479ff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.067825:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545479ff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.067833:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545479ff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.068076:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495917184:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43024us (43344us total) trans 4294973834 rc 0/0 00000100:00100000:1.0:1789760779.068087:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6546 00000100:00100000:1.0:1789760779.068098:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763312218100 time 763345751839 state 0x2 ref 0 00000001:00080000:0.0:1789760779.068660:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973834 is committed 00000100:00100000:1.0:1789760779.072121:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760779.072202:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495917312 00000100:00100000:3.0:1789760779.072212:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495917312 00000100:00100000:3.0:1789760779.072267:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763312218100 time 763345751839 state 0x0 ref 1 00000100:00100000:3.0:1789760779.072330:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763346012255 time 763345751839 state 0x1 ref 1 00000100:00100000:3.0:1789760779.072356:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763346012255 time 763393073101 state 0x0 ref 1 00000100:00100000:3.0:1789760779.072370:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495917312:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760779.072421:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26697728, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd009 00010000:00010000:3.0:1789760779.072430:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547a06 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.072445:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760779.072449:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.072455:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547a06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.072468:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547a06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.103086:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760779.106964:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26697728 -> 26701824 00010000:00010000:1.0:1789760779.106984:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547a06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.106997:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547a06 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.107010:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547a06 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.107017:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760779.107022:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547a06 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.107031:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547a06 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.107052:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547a06 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.107059:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547a06 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.107147:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495917312:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34775us (35031us total) trans 4294973835 rc 0/0 00000100:00100000:1.0:1789760779.107158:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6547 00000100:00100000:1.0:1789760779.107167:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763346012255 time 763393073101 state 0x2 ref 0 00000001:00080000:0.0:1789760779.107611:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973835 is committed 00000100:00100000:2.0:1789760779.115920:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760779.116044:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495917440 00000100:00100000:1.0:1789760779.116054:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495917440 00000100:00100000:1.0:1789760779.116106:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763346012255 time 763393073101 state 0x0 ref 1 00000100:00100000:1.0:1789760779.116122:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763393333517 time 763393073101 state 0x1 ref 1 00000100:00100000:1.0:1789760779.116157:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763393333517 time 763436873307 state 0x0 ref 1 00000100:00100000:1.0:1789760779.116170:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495917440:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760779.116220:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26701824, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd009 00010000:00010000:1.0:1789760779.116229:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354547a0d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.116246:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760779.116250:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.116257:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354547a0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.116271:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354547a0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.159520:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973836 is committed 00002000:00010000:1.0:1789760779.159639:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26701824 -> 26705920 00010000:00010000:1.0:1789760779.159655:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354547a0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.159667:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354547a0d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.159676:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354547a0d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.159684:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760779.159689:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354547a0d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.159698:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354547a0d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.159718:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354547a0d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.159726:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354547a0d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.159839:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495917440:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43668us (43925us total) trans 4294973836 rc 0/0 00000100:00100000:1.0:1789760779.159850:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6548 00000100:00100000:1.0:1789760779.159860:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763393333517 time 763436873307 state 0x2 ref 0 00000100:00100000:2.0:1789760779.165417:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760779.167036:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495917568 00000100:00100000:1.0:1789760779.167049:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495917568 00000100:00100000:1.0:1789760779.167956:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763393333517 time 763436873307 state 0x0 ref 1 00000100:00100000:1.0:1789760779.167974:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763437133723 time 763436873307 state 0x1 ref 1 00000100:00100000:1.0:1789760779.168002:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763437133723 time 763488719067 state 0x0 ref 1 00000100:00100000:1.0:1789760779.168017:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495917568:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760779.168065:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26705920, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd009 00010000:00010000:1.0:1789760779.168075:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547a14 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.168231:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760779.168236:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.168242:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547a14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.168257:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547a14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.236962:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973837 is committed 00002000:00010000:1.0:1789760779.238513:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26705920 -> 26710016 00010000:00010000:1.0:1789760779.238532:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547a14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.238546:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547a14 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.238555:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547a14 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.238563:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760779.238568:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547a14 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.238577:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547a14 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.238601:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547a14 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.238608:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547a14 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.238769:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495917568:12345-192.168.203.9@tcp:4:dd.0 Request processed in 70742us (73347us total) trans 4294973837 rc 0/0 00000100:00100000:1.0:1789760779.238790:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6549 00000100:00100000:1.0:1789760779.238801:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763437133723 time 763488719067 state 0x2 ref 0 00000100:00100000:0.0:1789760779.243615:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760779.243668:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495917696 00000100:00100000:1.0:1789760779.243678:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495917696 00000100:00100000:1.0:1789760779.243730:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763437133723 time 763488719067 state 0x0 ref 1 00000100:00100000:1.0:1789760779.243745:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763488979483 time 763488719067 state 0x1 ref 1 00000100:00100000:1.0:1789760779.243820:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763488979483 time 763564536664 state 0x0 ref 1 00000100:00100000:1.0:1789760779.243833:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495917696:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760779.243879:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26710016, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd009 00010000:00010000:1.0:1789760779.243888:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547a1b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.243903:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760779.243908:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.243915:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547a1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.243929:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547a1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.281096:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973838 is committed 00002000:00010000:3.0:1789760779.281198:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26710016 -> 26714112 00010000:00010000:3.0:1789760779.281214:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547a1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.281234:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547a1b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.281326:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547a1b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.281336:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760779.281341:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547a1b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.281351:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547a1b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.281371:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547a1b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.281378:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547a1b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.281470:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495917696:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37634us (37859us total) trans 4294973838 rc 0/0 00000100:00100000:3.0:1789760779.281480:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6550 00000100:00100000:3.0:1789760779.281491:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763488979483 time 763564536664 state 0x2 ref 0 00000100:00100000:2.0:1789760779.288824:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760779.288913:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495917824 00000100:00100000:1.0:1789760779.288924:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495917824 00000100:00100000:1.0:1789760779.288978:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763488979483 time 763564536664 state 0x0 ref 1 00000100:00100000:1.0:1789760779.288994:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763564797080 time 763564536664 state 0x1 ref 1 00000100:00100000:1.0:1789760779.289079:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763564797080 time 763609795359 state 0x0 ref 1 00000100:00100000:1.0:1789760779.289093:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495917824:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760779.289146:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26714112, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd009 00010000:00010000:1.0:1789760779.289155:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547a22 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.289171:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760779.289175:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.289181:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547a22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.289195:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547a22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.316470:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973839 is committed 00002000:00010000:1.0:1789760779.316580:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26714112 -> 26718208 00010000:00010000:1.0:1789760779.316597:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547a22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.316612:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547a22 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.316621:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547a22 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.316632:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760779.316637:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547a22 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.316647:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547a22 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.316670:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547a22 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.316678:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547a22 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.316793:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495917824:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27695us (27970us total) trans 4294973839 rc 0/0 00000100:00100000:1.0:1789760779.316809:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6551 00000100:00100000:1.0:1789760779.316819:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763564797080 time 763609795359 state 0x2 ref 0 00000100:00100000:1.0:1789760779.321139:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760779.321326:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495917952 00000100:00100000:0.0:1789760779.321337:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495917952 00000100:00100000:0.0:1789760779.321390:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763564797080 time 763609795359 state 0x0 ref 1 00000100:00100000:0.0:1789760779.321406:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763610055775 time 763609795359 state 0x1 ref 1 00000100:00100000:0.0:1789760779.321430:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763610055775 time 763642147420 state 0x0 ref 1 00000100:00100000:0.0:1789760779.321442:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495917952:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760779.321499:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26718208, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd009 00010000:00010000:0.0:1789760779.321511:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547a29 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.321528:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760779.321532:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.321537:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547a29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.321550:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547a29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.376018:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973840 is committed 00002000:00010000:1.0:1789760779.376089:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26718208 -> 26722304 00010000:00010000:1.0:1789760779.376105:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547a29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.376116:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547a29 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.376125:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547a29 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.376133:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760779.376137:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547a29 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.376147:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547a29 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.376376:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547a29 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.376384:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547a29 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.376517:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495917952:12345-192.168.203.9@tcp:4:dd.0 Request processed in 55072us (55385us total) trans 4294973840 rc 0/0 00000100:00100000:1.0:1789760779.376527:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6552 00000100:00100000:1.0:1789760779.376537:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763610055775 time 763642147420 state 0x2 ref 0 00000100:00100000:2.0:1789760779.384438:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760779.384525:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495918080 00000100:00100000:3.0:1789760779.384536:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495918080 00000100:00100000:3.0:1789760779.384593:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763610055775 time 763642147420 state 0x0 ref 1 00000100:00100000:3.0:1789760779.384608:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763642407836 time 763642147420 state 0x1 ref 1 00000100:00100000:3.0:1789760779.384639:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763642407836 time 763705355356 state 0x0 ref 1 00000100:00100000:3.0:1789760779.384652:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495918080:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760779.384698:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26722304, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd009 00010000:00010000:3.0:1789760779.384708:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547a30 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.384723:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760779.384728:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.384735:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547a30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.384749:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547a30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.414252:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973841 is committed 00002000:00010000:3.0:1789760779.414370:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26722304 -> 26726400 00010000:00010000:3.0:1789760779.414384:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547a30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.414396:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547a30 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.414410:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547a30 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.414492:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760779.414497:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547a30 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.414506:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547a30 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.414525:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547a30 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.414534:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547a30 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.414621:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495918080:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29967us (30187us total) trans 4294973841 rc 0/0 00000100:00100000:3.0:1789760779.414631:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6553 00000100:00100000:3.0:1789760779.414640:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763642407836 time 763705355356 state 0x2 ref 0 00000100:00100000:2.0:1789760779.420772:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760779.420862:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495918208 00000100:00100000:1.0:1789760779.420872:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495918208 00000100:00100000:1.0:1789760779.420938:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763642407836 time 763705355356 state 0x0 ref 1 00000100:00100000:1.0:1789760779.420955:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763705615772 time 763705355356 state 0x1 ref 1 00000100:00100000:1.0:1789760779.420983:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763705615772 time 763741699999 state 0x0 ref 1 00000100:00100000:1.0:1789760779.420995:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495918208:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760779.421041:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26726400, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd009 00010000:00010000:1.0:1789760779.421050:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547a37 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.421070:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760779.421082:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.421090:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547a37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.421103:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547a37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.460555:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973842 is committed 00002000:00010000:1.0:1789760779.460692:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26726400 -> 26730496 00010000:00010000:1.0:1789760779.460709:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547a37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.460723:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547a37 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.460733:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547a37 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.460749:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760779.460754:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547a37 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.460772:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547a37 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.460790:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547a37 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.460799:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547a37 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.461009:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495918208:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40012us (40252us total) trans 4294973842 rc 0/0 00000100:00100000:1.0:1789760779.461024:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6554 00000100:00100000:1.0:1789760779.461034:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763705615772 time 763741699999 state 0x2 ref 0 00000100:00100000:1.0:1789760779.464714:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760779.464974:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495918336 00000100:00100000:0.0:1789760779.464983:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495918336 00000100:00100000:0.0:1789760779.465038:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763705615772 time 763741699999 state 0x0 ref 1 00000100:00100000:0.0:1789760779.465053:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763741960415 time 763741699999 state 0x1 ref 1 00000100:00100000:0.0:1789760779.465094:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763741960415 time 763785810682 state 0x0 ref 1 00000100:00100000:0.0:1789760779.465111:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495918336:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760779.465158:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26730496, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd009 00010000:00010000:0.0:1789760779.465167:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547a3e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.465184:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760779.465189:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.465195:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547a3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.465209:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547a3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.509136:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973843 is committed 00002000:00010000:2.0:1789760779.509236:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26730496 -> 26734592 00010000:00010000:2.0:1789760779.509253:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547a3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.509266:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547a3e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.509306:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547a3e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.509314:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760779.509318:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547a3e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.509335:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547a3e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.509354:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547a3e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.509362:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547a3e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.509442:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495918336:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44329us (44732us total) trans 4294973843 rc 0/0 00000100:00100000:2.0:1789760779.509452:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6555 00000100:00100000:2.0:1789760779.509462:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763741960415 time 763785810682 state 0x2 ref 0 00000100:00100000:2.0:1789760779.517801:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760779.517889:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495918464 00000100:00100000:3.0:1789760779.517900:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495918464 00000100:00100000:3.0:1789760779.517955:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763741960415 time 763785810682 state 0x0 ref 1 00000100:00100000:3.0:1789760779.517970:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763786071098 time 763785810682 state 0x1 ref 1 00000100:00100000:3.0:1789760779.518001:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763786071098 time 763838717433 state 0x0 ref 1 00000100:00100000:3.0:1789760779.518014:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495918464:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760779.518060:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26734592, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd009 00010000:00010000:3.0:1789760779.518069:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547a45 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.518086:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760779.518210:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.518220:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547a45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.518234:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547a45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760779.549536:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26734592 -> 26738688 00010000:00010000:0.0:1789760779.549552:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547a45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.549562:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547a45 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.549568:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547a45 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.549572:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760779.549576:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547a45 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.549583:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547a45 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.549598:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547a45 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.549603:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547a45 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.549675:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495918464:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31658us (31886us total) trans 4294973844 rc 0/0 00000100:00100000:0.0:1789760779.549686:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6556 00000100:00100000:0.0:1789760779.549696:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763786071098 time 763838717433 state 0x2 ref 0 00000001:00080000:1.0:1789760779.549778:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973844 is committed 00000100:00100000:0.0:1789760779.554167:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760779.554263:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495918592 00000100:00100000:1.0:1789760779.554273:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495918592 00000100:00100000:1.0:1789760779.554399:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763786071098 time 763838717433 state 0x0 ref 1 00000100:00100000:1.0:1789760779.554415:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763838977849 time 763838717433 state 0x1 ref 1 00000100:00100000:1.0:1789760779.554484:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763838977849 time 763875200466 state 0x0 ref 1 00000100:00100000:1.0:1789760779.554497:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495918592:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760779.554543:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26738688, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd009 00010000:00010000:1.0:1789760779.554552:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547a4c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.554567:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760779.554571:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.554578:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547a4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.554592:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547a4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760779.582115:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26738688 -> 26742784 00010000:00010000:0.0:1789760779.582135:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547a4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.582148:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547a4c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.582156:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547a4c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.582169:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760779.582174:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547a4c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.582183:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547a4c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.582201:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547a4c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.582208:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547a4c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.582363:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495918592:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27863us (28200us total) trans 4294973845 rc 0/0 00000100:00100000:0.0:1789760779.582375:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6557 00000100:00100000:0.0:1789760779.582385:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763838977849 time 763875200466 state 0x2 ref 0 00000001:00080000:1.0:1789760779.583747:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973845 is committed 00000100:00100000:3.0:1789760779.589015:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760779.589119:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495918720 00000100:00100000:2.0:1789760779.589129:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495918720 00000100:00100000:2.0:1789760779.589190:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763838977849 time 763875200466 state 0x0 ref 1 00000100:00100000:2.0:1789760779.589210:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763875460882 time 763875200466 state 0x1 ref 1 00000100:00100000:2.0:1789760779.589246:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763875460882 time 763909962478 state 0x0 ref 1 00000100:00100000:2.0:1789760779.589258:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495918720:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760779.589351:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26742784, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd009 00010000:00010000:2.0:1789760779.589362:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547a53 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.589379:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:2.0:1789760779.589384:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.589390:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547a53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.589410:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547a53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.623298:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973846 is committed 00002000:00010000:3.0:1789760779.623470:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26742784 -> 26746880 00010000:00010000:3.0:1789760779.623487:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547a53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.623500:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547a53 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.623514:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547a53 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.623521:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760779.623526:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547a53 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.623535:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547a53 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.623556:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547a53 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.623563:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547a53 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.623654:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495918720:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34394us (34644us total) trans 4294973846 rc 0/0 00000100:00100000:3.0:1789760779.623664:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6558 00000100:00100000:3.0:1789760779.623674:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763875460882 time 763909962478 state 0x2 ref 0 00000100:00100000:3.0:1789760779.638003:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760779.638084:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495918848 00000100:00100000:0.0:1789760779.638095:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495918848 00000100:00100000:0.0:1789760779.638153:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763875460882 time 763909962478 state 0x0 ref 1 00000100:00100000:0.0:1789760779.638170:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763910222894 time 763909962478 state 0x1 ref 1 00000100:00100000:0.0:1789760779.638246:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763910222894 time 763958962101 state 0x0 ref 1 00000100:00100000:0.0:1789760779.638264:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495918848:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760779.638349:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26746880, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd009 00010000:00010000:0.0:1789760779.638358:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a5a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.638376:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760779.638382:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.638388:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.638403:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760779.666594:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26746880 -> 26750976 00002000:00010000:0.0:1789760779.666605:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760778 -> 1789760779 00002000:00010000:0.0:1789760779.666608:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760778 -> 1789760779 00010000:00010000:0.0:1789760779.666623:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.666635:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a5a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.666643:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a5a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.666655:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760779.666660:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a5a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.666669:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a5a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.666689:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a5a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.666695:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a5a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.666816:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495918848:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28551us (28817us total) trans 4294973847 rc 0/0 00000100:00100000:0.0:1789760779.666827:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6559 00000100:00100000:0.0:1789760779.666836:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763910222894 time 763958962101 state 0x2 ref 0 00000001:00080000:3.0:1789760779.667403:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973847 is committed 00000100:00100000:1.0:1789760779.675008:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760779.675188:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495918976 00000100:00100000:0.0:1789760779.675197:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495918976 00000100:00100000:0.0:1789760779.675252:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763910222894 time 763958962101 state 0x0 ref 1 00000100:00100000:0.0:1789760779.675267:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763959222517 time 763958962101 state 0x1 ref 1 00000100:00100000:0.0:1789760779.675355:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763959222517 time 763996071686 state 0x0 ref 1 00000100:00100000:0.0:1789760779.675372:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495918976:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760779.675425:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26750976, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd009 00010000:00010000:0.0:1789760779.675434:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a61 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.675451:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760779.675455:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.675461:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.675474:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.706375:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973848 is committed 00002000:00010000:0.0:1789760779.706426:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26750976 -> 26755072 00010000:00010000:0.0:1789760779.706441:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.706454:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a61 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.706464:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a61 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.706472:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760779.706477:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a61 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.706487:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a61 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.706518:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a61 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.706527:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a61 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.706634:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495918976:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31259us (31629us total) trans 4294973848 rc 0/0 00000100:00100000:0.0:1789760779.706644:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6560 00000100:00100000:0.0:1789760779.706654:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763959222517 time 763996071686 state 0x2 ref 0 00000100:00100000:1.0:1789760779.712188:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760779.712386:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495919104 00000100:00100000:3.0:1789760779.712397:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495919104 00000100:00100000:3.0:1789760779.712447:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763959222517 time 763996071686 state 0x0 ref 1 00000100:00100000:3.0:1789760779.712462:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763996332102 time 763996071686 state 0x1 ref 1 00000100:00100000:3.0:1789760779.712534:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763996332102 time 764033249444 state 0x0 ref 1 00000100:00100000:3.0:1789760779.712546:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495919104:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760779.712592:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26755072, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd009 00010000:00010000:3.0:1789760779.712601:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547a68 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.712617:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760779.712621:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.712627:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547a68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.712639:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547a68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.749383:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973849 is committed 00002000:00010000:0.0:1789760779.749486:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26755072 -> 26759168 00010000:00010000:0.0:1789760779.749502:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547a68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.749516:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547a68 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.749525:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547a68 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.749533:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760779.749538:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547a68 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.749548:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547a68 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.749567:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547a68 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.749575:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547a68 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.749672:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495919104:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37123us (37487us total) trans 4294973849 rc 0/0 00000100:00100000:0.0:1789760779.749682:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6561 00000100:00100000:0.0:1789760779.749693:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763996332102 time 764033249444 state 0x2 ref 0 00000100:00100000:3.0:1789760779.752969:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760779.753045:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495919232 00000100:00100000:0.0:1789760779.753056:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495919232 00000100:00100000:0.0:1789760779.753109:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 763996332102 time 764033249444 state 0x0 ref 1 00000100:00100000:0.0:1789760779.753123:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764033509860 time 764033249444 state 0x1 ref 1 00000100:00100000:0.0:1789760779.753143:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764033509860 time 764073859469 state 0x0 ref 1 00000100:00100000:0.0:1789760779.753154:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495919232:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760779.753203:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26759168, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd009 00010000:00010000:0.0:1789760779.753213:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a6f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.753234:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760779.753239:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.753246:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.754165:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.770363:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760779.770414:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760779.770452:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760779.770583:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495919360 00000100:00100000:0.0:1789760779.770593:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495919360 00000100:00100000:0.0:1789760779.770654:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 416 00000100:00100000:0.0:1789760779.770666:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290122ce450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2356:x1876699495919360:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:0.0:1789760779.770745:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290122ce450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2356:x1876699495919360:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 71us (321us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760779.770766:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 416 00000100:00100000:3.0:1789760779.771223:0:8003:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495919488 00000100:00100000:3.0:1789760779.771230:0:8003:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495919488 00000100:00100000:3.0:1789760779.771367:0:8003:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495919616 00000100:00100000:3.0:1789760779.771372:0:8003:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495919616 00000100:00100000:0.0:1789760779.771426:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 520 00000100:00100000:0.0:1789760779.771437:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:17022965-1e4f-46fb-915c-61730096b8e2+8:2356:x1876699495919488:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:0.0:1789760779.771501:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:17022965-1e4f-46fb-915c-61730096b8e2+8:2356:x1876699495919488:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 66us (1144us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760779.771508:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 520 00000100:00080000:1.0:1789760779.771878:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760779.772041:0:8003:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 521 00000100:00100000:3.0:1789760779.772049:0:8003:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2356:x1876699495919616:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:3.0:1789760779.772086:0:8003:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2356:x1876699495919616:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 38us (1634us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760779.772093:0:8003:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 521 00000001:00080000:1.0:1789760779.789748:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973850 is committed 00002000:00010000:0.0:1789760779.789899:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26759168 -> 26763264 00010000:00010000:0.0:1789760779.789915:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.789934:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a6f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.789942:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a6f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.789949:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760779.789954:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a6f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.789962:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a6f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.789981:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a6f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.789988:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547a6f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.790140:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495919232:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36920us (37113us total) trans 4294973850 rc 0/0 00000100:00100000:0.0:1789760779.790152:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6562 00000100:00100000:0.0:1789760779.790163:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764033509860 time 764073859469 state 0x2 ref 0 00000100:00100000:1.0:1789760779.793487:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760779.794640:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495919744 00000100:00100000:0.0:1789760779.794652:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495919744 00000100:00100000:1.0:1789760779.795647:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764033509860 time 764073859469 state 0x0 ref 1 00000100:00100000:1.0:1789760779.795731:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764074119885 time 764073859469 state 0x1 ref 1 00000100:00100000:1.0:1789760779.795778:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764074119885 time 764116494870 state 0x0 ref 1 00000100:00100000:1.0:1789760779.796019:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02768e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495919744:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760779.796070:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26763264, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd009 00010000:00010000:1.0:1789760779.796078:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354547a76 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.796092:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:1.0:1789760779.796097:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.796102:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354547a76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.796115:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354547a76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.826936:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973851 is committed 00002000:00010000:3.0:1789760779.827140:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26763264 -> 26767360 00010000:00010000:3.0:1789760779.827156:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354547a76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.827170:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354547a76 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.827180:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354547a76 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.827187:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760779.827192:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354547a76 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.827201:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354547a76 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.827221:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354547a76 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.827230:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x4936748354547a76 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.827394:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02768e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495919744:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31601us (33910us total) trans 4294973851 rc 0/0 00000100:00100000:3.0:1789760779.827405:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6563 00000100:00100000:3.0:1789760779.827413:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764074119885 time 764116494870 state 0x2 ref 0 00000100:00100000:1.0:1789760779.832719:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760779.832837:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495919872 00000100:00100000:0.0:1789760779.832847:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495919872 00000100:00100000:0.0:1789760779.833024:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764074119885 time 764116494870 state 0x0 ref 1 00000100:00100000:0.0:1789760779.833038:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764116755286 time 764116494870 state 0x1 ref 1 00000100:00100000:0.0:1789760779.833070:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764116755286 time 764153786112 state 0x0 ref 1 00000100:00100000:0.0:1789760779.833084:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495919872:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760779.833130:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26767360, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd009 00010000:00010000:0.0:1789760779.833138:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547a7d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.833154:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760779.833159:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.833166:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547a7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.833181:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547a7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.875259:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973852 is committed 00002000:00010000:1.0:1789760779.875384:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26767360 -> 26771456 00010000:00010000:1.0:1789760779.875399:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547a7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.875412:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547a7d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.875421:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547a7d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.875429:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760779.875434:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547a7d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.875444:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547a7d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.875464:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547a7d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.875472:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547a7d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.875565:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495919872:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42479us (42852us total) trans 4294973852 rc 0/0 00000100:00100000:1.0:1789760779.875586:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6564 00000100:00100000:1.0:1789760779.875597:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764116755286 time 764153786112 state 0x2 ref 0 00000100:00100000:0.0:1789760779.878577:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760779.878869:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495920000 00000100:00100000:3.0:1789760779.878879:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495920000 00000100:00100000:3.0:1789760779.878934:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764116755286 time 764153786112 state 0x0 ref 1 00000100:00100000:3.0:1789760779.878949:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764154046528 time 764153786112 state 0x1 ref 1 00000100:00100000:3.0:1789760779.878981:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764154046528 time 764199697109 state 0x0 ref 1 00000100:00100000:3.0:1789760779.878996:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04575500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495920000:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760779.879179:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26771456, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd009 00010000:00010000:3.0:1789760779.879190:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a84 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.879206:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760779.879211:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.879218:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.879232:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760779.916275:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26771456 -> 26775552 00010000:00010000:3.0:1789760779.916331:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.916344:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a84 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.916354:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a84 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.916361:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760779.916366:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a84 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.916375:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a84 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.916394:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a84 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.916402:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a84 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.916485:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04575500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495920000:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37488us (37910us total) trans 4294973853 rc 0/0 00000100:00100000:3.0:1789760779.916498:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6565 00000100:00100000:3.0:1789760779.916508:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764154046528 time 764199697109 state 0x2 ref 0 00000001:00080000:3.0:1789760779.916554:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973853 is committed 00000100:00100000:0.0:1789760779.922235:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760779.922360:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495920128 00000100:00100000:3.0:1789760779.922371:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495920128 00000100:00100000:3.0:1789760779.922510:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764154046528 time 764199697109 state 0x0 ref 1 00000100:00100000:3.0:1789760779.922527:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764199957525 time 764199697109 state 0x1 ref 1 00000100:00100000:3.0:1789760779.922611:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764199957525 time 764243327212 state 0x0 ref 1 00000100:00100000:3.0:1789760779.922625:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04577b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495920128:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760779.922675:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26775552, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd009 00010000:00010000:3.0:1789760779.922684:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a8b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.922700:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760779.922705:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.922713:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.922727:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.960726:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973854 is committed 00002000:00010000:3.0:1789760779.961075:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26775552 -> 26779648 00010000:00010000:3.0:1789760779.961091:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.961105:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a8b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.961114:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.961122:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760779.961127:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.961136:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a8b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.961156:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a8b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.961167:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a8b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.961321:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04577b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495920128:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38694us (39091us total) trans 4294973854 rc 0/0 00000100:00100000:3.0:1789760779.961332:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6566 00000100:00100000:3.0:1789760779.961342:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764199957525 time 764243327212 state 0x2 ref 0 00000100:00100000:0.0:1789760779.964939:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760779.965007:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495920256 00000100:00100000:1.0:1789760779.965021:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495920256 00000100:00100000:1.0:1789760779.965082:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764199957525 time 764243327212 state 0x0 ref 1 00000100:00100000:1.0:1789760779.965097:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764243587628 time 764243327212 state 0x1 ref 1 00000100:00100000:1.0:1789760779.965120:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764243587628 time 764285837859 state 0x0 ref 1 00000100:00100000:1.0:1789760779.965131:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01c8d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495920256:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760779.965180:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26779648, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd009 00010000:00010000:1.0:1789760779.965189:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547a92 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.965206:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:1.0:1789760779.965210:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.965217:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547a92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.965231:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547a92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.000606:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973855 is committed 00002000:00010000:0.0:1789760780.000694:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26779648 -> 26783744 00010000:00010000:0.0:1789760780.000711:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547a92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.000730:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547a92 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.000859:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547a92 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.000868:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760780.000873:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547a92 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.000882:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547a92 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.000902:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547a92 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.000910:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547a92 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.001169:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01c8d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495920256:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36034us (36234us total) trans 4294973855 rc 0/0 00000100:00100000:0.0:1789760780.001181:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6567 00000100:00100000:0.0:1789760780.001191:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764243587628 time 764285837859 state 0x2 ref 0 00000100:00100000:0.0:1789760780.006892:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760780.009885:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495920384 00000100:00100000:3.0:1789760780.009901:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495920384 00000100:00100000:3.0:1789760780.009965:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764243587628 time 764285837859 state 0x0 ref 1 00000100:00100000:3.0:1789760780.009981:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764286098275 time 764285837859 state 0x1 ref 1 00000100:00100000:3.0:1789760780.010972:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764286098275 time 764331681868 state 0x0 ref 1 00000100:00100000:3.0:1789760780.010994:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495920384:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760780.011047:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26783744, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd009 00010000:00010000:3.0:1789760780.011057:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a99 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.011074:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760780.011079:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.011086:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.011099:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.054176:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973856 is committed 00002000:00010000:3.0:1789760780.054507:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26783744 -> 26787840 00010000:00010000:3.0:1789760780.054523:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.054535:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a99 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.054544:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.054551:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760780.054555:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.054563:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a99 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.054582:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a99 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.054591:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547a99 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.056380:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495920384:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45382us (49485us total) trans 4294973856 rc 0/0 00000100:00100000:3.0:1789760780.056394:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6568 00000100:00100000:3.0:1789760780.056405:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764286098275 time 764331681868 state 0x2 ref 0 00000100:00100000:1.0:1789760780.060738:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760780.060834:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495920512 00000100:00100000:3.0:1789760780.060844:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495920512 00000100:00100000:3.0:1789760780.060965:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764286098275 time 764331681868 state 0x0 ref 1 00000100:00100000:3.0:1789760780.060979:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764331942284 time 764331681868 state 0x1 ref 1 00000100:00100000:3.0:1789760780.061043:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764331942284 time 764381759283 state 0x0 ref 1 00000100:00100000:3.0:1789760780.061054:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04929500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495920512:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760780.061099:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26787840, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd009 00010000:00010000:3.0:1789760780.061107:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aa0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.061123:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760780.061127:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.061134:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aa0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.061148:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aa0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.090035:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973857 is committed 00002000:00010000:3.0:1789760780.090143:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26787840 -> 26791936 00010000:00010000:3.0:1789760780.090159:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aa0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.090172:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aa0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.090181:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aa0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.090188:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760780.090194:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aa0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.090203:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aa0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.090224:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aa0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.090232:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aa0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.090408:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04929500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495920512:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29350us (29673us total) trans 4294973857 rc 0/0 00000100:00100000:3.0:1789760780.090418:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6569 00000100:00100000:3.0:1789760780.090428:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764331942284 time 764381759283 state 0x2 ref 0 00000100:00100000:1.0:1789760780.094700:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760780.095034:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495920640 00000100:00100000:0.0:1789760780.095044:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495920640 00000100:00100000:0.0:1789760780.095100:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764331942284 time 764381759283 state 0x0 ref 1 00000100:00100000:0.0:1789760780.095115:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764382019699 time 764381759283 state 0x1 ref 1 00000100:00100000:0.0:1789760780.095432:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764382019699 time 764416142634 state 0x0 ref 1 00000100:00100000:0.0:1789760780.095447:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495920640:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760780.095494:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26791936, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd009 00010000:00010000:0.0:1789760780.095504:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547aa7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.095520:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760780.095524:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.095531:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547aa7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.095545:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547aa7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.127347:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760780.128355:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973858 is committed 00002000:00010000:1.0:1789760780.128479:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26791936 -> 26796032 00010000:00010000:1.0:1789760780.128494:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547aa7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.128507:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547aa7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.128515:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547aa7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.128522:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760780.128527:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547aa7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.128535:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547aa7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.128554:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547aa7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.128561:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547aa7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.128658:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495920640:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33209us (33965us total) trans 4294973858 rc 0/0 00000100:00100000:1.0:1789760780.128668:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6570 00000100:00100000:1.0:1789760780.128678:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764382019699 time 764416142634 state 0x2 ref 0 00000100:00100000:1.0:1789760780.134438:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760780.134543:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495920768 00000100:00100000:3.0:1789760780.134554:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495920768 00000100:00100000:3.0:1789760780.134605:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764382019699 time 764416142634 state 0x0 ref 1 00000100:00100000:3.0:1789760780.134620:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764416403050 time 764416142634 state 0x1 ref 1 00000100:00100000:3.0:1789760780.134651:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764416403050 time 764455367693 state 0x0 ref 1 00000100:00100000:3.0:1789760780.134663:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495920768:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760780.134712:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26796032, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd009 00010000:00010000:3.0:1789760780.134721:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.134748:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760780.134761:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.134767:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.134780:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.171192:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973859 is committed 00002000:00010000:3.0:1789760780.171322:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26796032 -> 26800128 00010000:00010000:3.0:1789760780.171338:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.171351:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.171361:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.171369:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760780.171374:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.171382:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.171401:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.171409:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.171494:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495920768:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36828us (37060us total) trans 4294973859 rc 0/0 00000100:00100000:3.0:1789760780.171505:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6571 00000100:00100000:3.0:1789760780.171515:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764416403050 time 764455367693 state 0x2 ref 0 00000100:00100000:0.0:1789760780.175875:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760780.175943:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495920896 00000100:00100000:3.0:1789760780.175953:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495920896 00000100:00100000:3.0:1789760780.176007:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764416403050 time 764455367693 state 0x0 ref 1 00000100:00100000:3.0:1789760780.176022:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764455628109 time 764455367693 state 0x1 ref 1 00000100:00100000:3.0:1789760780.176051:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764455628109 time 764496768810 state 0x0 ref 1 00000100:00100000:3.0:1789760780.176063:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495920896:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760780.176107:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26800128, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd009 00010000:00010000:3.0:1789760780.176116:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547ab5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.176132:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760780.176191:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.176199:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547ab5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.176214:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547ab5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.217129:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973860 is committed 00002000:00010000:1.0:1789760780.217200:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26800128 -> 26804224 00010000:00010000:1.0:1789760780.217231:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547ab5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.217246:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547ab5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.217255:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547ab5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.217264:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760780.217269:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547ab5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.217314:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547ab5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.217335:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547ab5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.217349:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547ab5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.217890:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495920896:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41823us (42018us total) trans 4294973860 rc 0/0 00000100:00100000:1.0:1789760780.217902:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6572 00000100:00100000:1.0:1789760780.217912:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764455628109 time 764496768810 state 0x2 ref 0 00000100:00100000:1.0:1789760780.228451:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760780.228549:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495921024 00000100:00100000:0.0:1789760780.228559:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495921024 00000100:00100000:0.0:1789760780.228615:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764455628109 time 764496768810 state 0x0 ref 1 00000100:00100000:0.0:1789760780.228631:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764497029226 time 764496768810 state 0x1 ref 1 00000100:00100000:0.0:1789760780.229080:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764497029226 time 764549795630 state 0x0 ref 1 00000100:00100000:0.0:1789760780.229095:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495921024:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760780.229141:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26804224, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd009 00010000:00010000:0.0:1789760780.229150:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547abc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.229166:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760780.229170:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.229176:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547abc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.229191:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547abc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.273447:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973861 is committed 00002000:00010000:1.0:1789760780.273636:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26804224 -> 26808320 00010000:00010000:1.0:1789760780.273653:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547abc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.273665:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547abc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.273675:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547abc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.273682:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760780.273687:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547abc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.273696:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547abc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.273720:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547abc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.273725:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547abc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.273826:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495921024:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44730us (45379us total) trans 4294973861 rc 0/0 00000100:00100000:1.0:1789760780.273836:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6573 00000100:00100000:1.0:1789760780.273845:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764497029226 time 764549795630 state 0x2 ref 0 00000100:00100000:0.0:1789760780.278194:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760780.278271:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495921152 00000100:00100000:3.0:1789760780.278311:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495921152 00000100:00100000:3.0:1789760780.278365:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764497029226 time 764549795630 state 0x0 ref 1 00000100:00100000:3.0:1789760780.278382:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764550056046 time 764549795630 state 0x1 ref 1 00000100:00100000:3.0:1789760780.278420:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764550056046 time 764599136451 state 0x0 ref 1 00000100:00100000:3.0:1789760780.278433:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495921152:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760780.278479:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26808320, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd009 00010000:00010000:3.0:1789760780.278487:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547ac3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.278503:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760780.278508:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.278514:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547ac3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.278527:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547ac3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.305056:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973862 is committed 00002000:00010000:3.0:1789760780.305184:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26808320 -> 26812416 00010000:00010000:3.0:1789760780.305199:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547ac3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.305212:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547ac3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.305222:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547ac3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.305229:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760780.305234:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547ac3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.305243:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547ac3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.305262:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547ac3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.305270:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547ac3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.305469:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495921152:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27032us (27277us total) trans 4294973862 rc 0/0 00000100:00100000:3.0:1789760780.305483:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6574 00000100:00100000:3.0:1789760780.305493:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764550056046 time 764599136451 state 0x2 ref 0 00000100:00100000:0.0:1789760780.311258:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760780.311685:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495921280 00000100:00100000:1.0:1789760780.311695:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495921280 00000100:00100000:1.0:1789760780.311769:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764550056046 time 764599136451 state 0x0 ref 1 00000100:00100000:1.0:1789760780.311783:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764599396867 time 764599136451 state 0x1 ref 1 00000100:00100000:1.0:1789760780.311816:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764599396867 time 764632533127 state 0x0 ref 1 00000100:00100000:1.0:1789760780.311828:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495921280:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760780.311874:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26812416, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd009 00010000:00010000:1.0:1789760780.311883:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547aca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.311898:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:1.0:1789760780.311903:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.311909:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547aca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.311922:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547aca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760780.344389:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26812416 -> 26816512 00010000:00010000:1.0:1789760780.344408:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547aca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.344422:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547aca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.344431:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547aca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.344439:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760780.344444:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547aca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.344453:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547aca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.344473:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547aca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.344481:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547aca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.344576:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495921280:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32745us (33322us total) trans 4294973863 rc 0/0 00000100:00100000:1.0:1789760780.344587:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6575 00000100:00100000:1.0:1789760780.344598:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764599396867 time 764632533127 state 0x2 ref 0 00000001:00080000:3.0:1789760780.344684:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973863 is committed 00000100:00100000:1.0:1789760780.350810:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760780.350968:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495921408 00000100:00100000:0.0:1789760780.350978:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495921408 00000100:00100000:0.0:1789760780.351033:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764599396867 time 764632533127 state 0x0 ref 1 00000100:00100000:0.0:1789760780.351048:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764632793543 time 764632533127 state 0x1 ref 1 00000100:00100000:0.0:1789760780.351084:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764632793543 time 764671800630 state 0x0 ref 1 00000100:00100000:0.0:1789760780.351101:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495921408:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760780.351151:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26816512, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd009 00010000:00010000:0.0:1789760780.351161:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547ad1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.351178:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760780.351183:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.351190:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547ad1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.351203:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547ad1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.380277:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973864 is committed 00002000:00010000:1.0:1789760780.380434:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26816512 -> 26820608 00010000:00010000:1.0:1789760780.380450:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547ad1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.380463:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547ad1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.380473:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547ad1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.380481:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760780.380486:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547ad1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.380495:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547ad1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.380515:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547ad1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.380527:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354547ad1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.380623:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495921408:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29519us (29817us total) trans 4294973864 rc 0/0 00000100:00100000:1.0:1789760780.380634:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6576 00000100:00100000:1.0:1789760780.380643:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764632793543 time 764671800630 state 0x2 ref 0 00000100:00100000:2.0:1789760780.385133:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760780.385218:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495921536 00000100:00100000:3.0:1789760780.385227:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495921536 00000100:00100000:3.0:1789760780.385337:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764632793543 time 764671800630 state 0x0 ref 1 00000100:00100000:3.0:1789760780.385354:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764672061046 time 764671800630 state 0x1 ref 1 00000100:00100000:3.0:1789760780.385377:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764672061046 time 764706094439 state 0x0 ref 1 00000100:00100000:3.0:1789760780.385390:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495921536:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760780.385435:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26820608, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd009 00010000:00010000:3.0:1789760780.385444:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547ad8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.385460:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760780.385465:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.385472:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547ad8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.385485:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547ad8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760780.422303:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26820608 -> 26824704 00010000:00010000:3.0:1789760780.422326:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547ad8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.422339:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547ad8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.422348:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547ad8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.422355:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760780.422360:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547ad8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.422368:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547ad8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.422388:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547ad8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.422396:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547ad8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.422489:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495921536:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37097us (37361us total) trans 4294973865 rc 0/0 00000100:00100000:3.0:1789760780.422500:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6577 00000100:00100000:3.0:1789760780.422510:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764672061046 time 764706094439 state 0x2 ref 0 00000001:00080000:2.0:1789760780.423049:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973865 is committed 00000100:00100000:2.0:1789760780.428377:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760780.428561:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495921664 00000100:00100000:1.0:1789760780.428571:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495921664 00000100:00100000:1.0:1789760780.428627:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764672061046 time 764706094439 state 0x0 ref 1 00000100:00100000:1.0:1789760780.428642:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764706354855 time 764706094439 state 0x1 ref 1 00000100:00100000:1.0:1789760780.428759:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764706354855 time 764749464827 state 0x0 ref 1 00000100:00100000:1.0:1789760780.428774:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495921664:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760780.428820:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26824704, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd009 00010000:00010000:1.0:1789760780.428829:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547adf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.428897:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:1.0:1789760780.428901:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.428907:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547adf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.428924:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547adf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.464372:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973866 is committed 00002000:00010000:1.0:1789760780.464447:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26824704 -> 26828800 00010000:00010000:1.0:1789760780.464463:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547adf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.464477:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547adf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.464487:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547adf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.464494:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760780.464499:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547adf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.464509:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547adf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.464528:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547adf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.464536:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547adf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.464634:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495921664:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35858us (36260us total) trans 4294973866 rc 0/0 00000100:00100000:1.0:1789760780.464645:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6578 00000100:00100000:1.0:1789760780.464655:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764706354855 time 764749464827 state 0x2 ref 0 00000100:00100000:1.0:1789760780.471791:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760780.471871:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495921792 00000100:00100000:0.0:1789760780.471882:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495921792 00000100:00100000:0.0:1789760780.471936:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764706354855 time 764749464827 state 0x0 ref 1 00000100:00100000:0.0:1789760780.471952:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764749725243 time 764749464827 state 0x1 ref 1 00000100:00100000:0.0:1789760780.471986:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764749725243 time 764792699069 state 0x0 ref 1 00000100:00100000:0.0:1789760780.472000:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495921792:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760780.472049:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26828800, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd009 00010000:00010000:0.0:1789760780.472058:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547ae6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.472076:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760780.472081:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.472088:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547ae6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.472102:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547ae6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760780.504075:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26828800 -> 26832896 00010000:00010000:0.0:1789760780.504096:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547ae6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.504109:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547ae6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.504117:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547ae6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.504125:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760780.504129:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547ae6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.504138:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547ae6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.504159:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547ae6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.504166:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547ae6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.504332:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495921792:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32330us (32545us total) trans 4294973867 rc 0/0 00000100:00100000:0.0:1789760780.504343:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6579 00000100:00100000:0.0:1789760780.504354:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764749725243 time 764792699069 state 0x2 ref 0 00000001:00080000:2.0:1789760780.504438:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973867 is committed 00000100:00100000:2.0:1789760780.509180:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760780.509433:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495921920 00000100:00100000:3.0:1789760780.509442:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495921920 00000100:00100000:3.0:1789760780.509486:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764749725243 time 764792699069 state 0x0 ref 1 00000100:00100000:3.0:1789760780.509497:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764792959485 time 764792699069 state 0x1 ref 1 00000100:00100000:3.0:1789760780.509518:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764792959485 time 764830236611 state 0x0 ref 1 00000100:00100000:3.0:1789760780.509527:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036398050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495921920:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760780.509569:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26832896, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd009 00010000:00010000:3.0:1789760780.509578:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.509594:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760780.509598:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.509605:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.509624:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.531495:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973868 is committed 00002000:00010000:1.0:1789760780.532468:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26832896 -> 26836992 00010000:00010000:1.0:1789760780.532485:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.532498:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.532507:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.532515:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760780.532520:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.532529:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.532623:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.532644:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547aed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.532825:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036398050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495921920:12345-192.168.203.9@tcp:4:dd.0 Request processed in 23292us (23649us total) trans 4294973868 rc 0/0 00000100:00100000:1.0:1789760780.532835:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6580 00000100:00100000:1.0:1789760780.532844:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764792959485 time 764830236611 state 0x2 ref 0 00000100:00100000:1.0:1789760780.537878:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760780.539175:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495922048 00000100:00100000:3.0:1789760780.539187:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495922048 00000100:00100000:3.0:1789760780.539253:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764792959485 time 764830236611 state 0x0 ref 1 00000100:00100000:3.0:1789760780.539268:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764830497027 time 764830236611 state 0x1 ref 1 00000100:00100000:0.0:1789760780.539346:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764830497027 time 764860059346 state 0x0 ref 1 00000100:00100000:0.0:1789760780.539363:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495922048:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760780.539422:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26836992, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd009 00010000:00010000:0.0:1789760780.539431:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547af4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.539446:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760780.539451:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.539457:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547af4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.539471:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547af4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.564689:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973869 is committed 00002000:00010000:1.0:1789760780.564886:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26836992 -> 26841088 00010000:00010000:1.0:1789760780.564904:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547af4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.564917:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547af4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.564926:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547af4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.566031:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760780.566039:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547af4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.566050:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547af4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.566150:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547af4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.566161:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547af4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.566349:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495922048:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26985us (28474us total) trans 4294973869 rc 0/0 00000100:00100000:1.0:1789760780.566359:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6581 00000100:00100000:1.0:1789760780.566369:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764830497027 time 764860059346 state 0x2 ref 0 00000100:00100000:2.0:1789760780.573253:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760780.573498:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495922176 00000100:00100000:0.0:1789760780.573508:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495922176 00000100:00100000:0.0:1789760780.573646:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764830497027 time 764860059346 state 0x0 ref 1 00000100:00100000:0.0:1789760780.573660:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764860319762 time 764860059346 state 0x1 ref 1 00000100:00100000:0.0:1789760780.573690:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764860319762 time 764894406753 state 0x0 ref 1 00000100:00100000:0.0:1789760780.573703:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495922176:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760780.573747:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26841088, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd009 00010000:00010000:0.0:1789760780.573768:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547afb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.573791:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760780.573796:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.573802:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547afb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.573816:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547afb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760780.604609:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26841088 -> 26845184 00010000:00010000:1.0:1789760780.604629:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547afb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.604643:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547afb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.604652:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547afb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.604659:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760780.604664:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547afb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.604678:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547afb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.604697:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547afb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.604704:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547afb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.604816:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495922176:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31110us (31566us total) trans 4294973870 rc 0/0 00000100:00100000:1.0:1789760780.604830:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6582 00000100:00100000:1.0:1789760780.604841:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764860319762 time 764894406753 state 0x2 ref 0 00000001:00080000:2.0:1789760780.607677:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973870 is committed 00000100:00100000:3.0:1789760780.607843:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760780.607991:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495922304 00000100:00100000:1.0:1789760780.608001:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495922304 00000100:00100000:1.0:1789760780.608057:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764860319762 time 764894406753 state 0x0 ref 1 00000100:00100000:1.0:1789760780.608073:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764894667169 time 764894406753 state 0x1 ref 1 00000100:00100000:1.0:1789760780.608100:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764894667169 time 764928817622 state 0x0 ref 1 00000100:00100000:1.0:1789760780.608113:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495922304:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760780.608160:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26845184, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd009 00010000:00010000:1.0:1789760780.608170:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547b02 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.608250:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:1.0:1789760780.608253:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.608259:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547b02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.608270:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547b02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.631264:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973871 is committed 00002000:00010000:1.0:1789760780.631534:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26845184 -> 26849280 00010000:00010000:1.0:1789760780.631553:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547b02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.631568:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547b02 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.631578:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547b02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.631587:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760780.631592:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547b02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.631602:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547b02 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.631630:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547b02 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.631639:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547b02 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.631781:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495922304:12345-192.168.203.9@tcp:4:dd.0 Request processed in 23665us (23937us total) trans 4294973871 rc 0/0 00000100:00100000:1.0:1789760780.631792:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6583 00000100:00100000:1.0:1789760780.631807:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764894667169 time 764928817622 state 0x2 ref 0 00000100:00100000:3.0:1789760780.636136:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760780.636599:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495922432 00000100:00100000:1.0:1789760780.636610:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495922432 00000100:00100000:1.0:1789760780.636672:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764894667169 time 764928817622 state 0x0 ref 1 00000100:00100000:1.0:1789760780.636687:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764929078038 time 764928817622 state 0x1 ref 1 00000100:00100000:1.0:1789760780.636723:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764929078038 time 764957439885 state 0x0 ref 1 00000100:00100000:1.0:1789760780.636746:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495922432:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760780.636805:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26849280, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd009 00010000:00010000:1.0:1789760780.636814:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547b09 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.636831:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:1.0:1789760780.636837:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.636844:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547b09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.636867:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547b09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.666438:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973872 is committed 00002000:00010000:0.0:1789760780.666537:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26849280 -> 26853376 00002000:00010000:0.0:1789760780.666542:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760779 -> 1789760780 00002000:00010000:0.0:1789760780.666545:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760779 -> 1789760780 00010000:00010000:0.0:1789760780.666560:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547b09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.666573:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547b09 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.666583:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547b09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.666591:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760780.666596:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547b09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.666606:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547b09 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.666626:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547b09 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.666634:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547b09 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.666742:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495922432:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29993us (30616us total) trans 4294973872 rc 0/0 00000100:00100000:0.0:1789760780.666761:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6584 00000100:00100000:0.0:1789760780.666771:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764929078038 time 764957439885 state 0x2 ref 0 00000100:00100000:2.0:1789760780.670961:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760780.671055:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495922560 00000100:00100000:1.0:1789760780.671065:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495922560 00000100:00100000:1.0:1789760780.671134:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764929078038 time 764957439885 state 0x0 ref 1 00000100:00100000:1.0:1789760780.671150:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764957700301 time 764957439885 state 0x1 ref 1 00000100:00100000:1.0:1789760780.671188:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764957700301 time 764991896887 state 0x0 ref 1 00000100:00100000:1.0:1789760780.671201:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495922560:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760780.671251:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26853376, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd009 00010000:00010000:1.0:1789760780.671260:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547b10 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.671276:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:1.0:1789760780.671324:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.671331:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547b10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.671345:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547b10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760780.699635:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26853376 -> 26857472 00010000:00010000:0.0:1789760780.699660:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547b10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.699679:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547b10 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.699688:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547b10 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.699695:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760780.699711:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547b10 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.699723:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547b10 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.699770:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547b10 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.699779:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547b10 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.699908:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495922560:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28706us (28952us total) trans 4294973873 rc 0/0 00000100:00100000:0.0:1789760780.699919:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6585 00000100:00100000:0.0:1789760780.699929:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764957700301 time 764991896887 state 0x2 ref 0 00000001:00080000:1.0:1789760780.700061:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973873 is committed 00000100:00100000:3.0:1789760780.707492:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760780.707597:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495922688 00000100:00100000:1.0:1789760780.707607:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495922688 00000100:00100000:1.0:1789760780.707657:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764957700301 time 764991896887 state 0x0 ref 1 00000100:00100000:1.0:1789760780.707683:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764992157303 time 764991896887 state 0x1 ref 1 00000100:00100000:1.0:1789760780.707712:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764992157303 time 765028428834 state 0x0 ref 1 00000100:00100000:1.0:1789760780.707725:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495922688:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760780.707780:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26857472, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd009 00010000:00010000:1.0:1789760780.707789:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547b17 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.707803:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760780.707808:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.707815:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547b17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.707829:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547b17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760780.734085:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26857472 -> 26861568 00010000:00010000:1.0:1789760780.734105:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547b17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.734125:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547b17 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.734134:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547b17 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.734144:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760780.734148:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547b17 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.734157:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547b17 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.734176:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547b17 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.734184:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547b17 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.734308:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495922688:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26549us (26788us total) trans 4294973874 rc 0/0 00000100:00100000:1.0:1789760780.734319:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6586 00000100:00100000:1.0:1789760780.734331:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764992157303 time 765028428834 state 0x2 ref 0 00000001:00080000:0.0:1789760780.734406:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973874 is committed 00000100:00100000:3.0:1789760780.739176:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760780.739601:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495922816 00000100:00100000:0.0:1789760780.739611:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495922816 00000100:00100000:0.0:1789760780.739668:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 764992157303 time 765028428834 state 0x0 ref 1 00000100:00100000:0.0:1789760780.739684:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765028689250 time 765028428834 state 0x1 ref 1 00000100:00100000:0.0:1789760780.739923:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765028689250 time 765060637489 state 0x0 ref 1 00000100:00100000:0.0:1789760780.739938:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495922816:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760780.739988:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26861568, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd009 00010000:00010000:0.0:1789760780.739997:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b1e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.740014:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760780.740018:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.740024:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.740038:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.774793:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973875 is committed 00002000:00010000:0.0:1789760780.774999:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26861568 -> 26865664 00010000:00010000:0.0:1789760780.775014:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.775026:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b1e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.775039:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b1e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.775047:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760780.775052:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b1e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.775060:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b1e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.775080:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b1e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.775087:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b1e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.775222:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495922816:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35282us (36049us total) trans 4294973875 rc 0/0 00000100:00100000:0.0:1789760780.775232:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6587 00000100:00100000:0.0:1789760780.775242:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765028689250 time 765060637489 state 0x2 ref 0 00000100:00100000:3.0:1789760780.788382:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760780.791637:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495922944 00000100:00100000:0.0:1789760780.791651:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495922944 00000100:00100000:0.0:1789760780.791717:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765028689250 time 765060637489 state 0x0 ref 1 00000100:00100000:0.0:1789760780.791732:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765060897905 time 765060637489 state 0x1 ref 1 00000100:00100000:0.0:1789760780.791776:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765060897905 time 765112493272 state 0x0 ref 1 00000100:00100000:0.0:1789760780.791790:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495922944:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760780.791842:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26865664, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd009 00010000:00010000:0.0:1789760780.791851:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b25 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.791867:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760780.791872:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.791878:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.791901:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.846856:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973876 is committed 00002000:00010000:0.0:1789760780.847889:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26865664 -> 26869760 00010000:00010000:0.0:1789760780.847907:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.847920:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b25 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.847930:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.847937:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760780.847942:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.847950:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b25 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.847968:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b25 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.847975:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b25 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.848143:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495922944:12345-192.168.203.9@tcp:4:dd.0 Request processed in 56281us (59695us total) trans 4294973876 rc 0/0 00000100:00100000:0.0:1789760780.848153:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6588 00000100:00100000:0.0:1789760780.848162:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765060897905 time 765112493272 state 0x2 ref 0 00000100:00100000:3.0:1789760780.852988:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760780.853186:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495923072 00000100:00100000:1.0:1789760780.853196:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495923072 00000100:00100000:1.0:1789760780.853252:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765060897905 time 765112493272 state 0x0 ref 1 00000100:00100000:1.0:1789760780.853267:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765112753688 time 765112493272 state 0x1 ref 1 00000100:00100000:1.0:1789760780.853339:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765112753688 time 765174055520 state 0x0 ref 1 00000100:00100000:1.0:1789760780.853352:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495923072:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760780.853401:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26869760, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd009 00010000:00010000:1.0:1789760780.853410:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547b2c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.853426:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760780.853431:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.853438:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547b2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.853452:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547b2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.882594:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973877 is committed 00002000:00010000:0.0:1789760780.883029:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26869760 -> 26873856 00010000:00010000:0.0:1789760780.883043:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547b2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.883055:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547b2c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.883065:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547b2c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.883079:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760780.883084:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547b2c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.883094:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547b2c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.883113:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547b2c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.883122:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547b2c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.883374:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495923072:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30018us (30389us total) trans 4294973877 rc 0/0 00000100:00100000:0.0:1789760780.883384:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6589 00000100:00100000:0.0:1789760780.883394:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765112753688 time 765174055520 state 0x2 ref 0 00000100:00100000:3.0:1789760780.887519:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760780.887982:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495923200 00000100:00100000:0.0:1789760780.887991:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495923200 00000100:00100000:0.0:1789760780.888044:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765112753688 time 765174055520 state 0x0 ref 1 00000100:00100000:0.0:1789760780.888059:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765174315936 time 765174055520 state 0x1 ref 1 00000100:00100000:0.0:1789760780.888178:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765174315936 time 765208893709 state 0x0 ref 1 00000100:00100000:0.0:1789760780.888195:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495923200:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760780.888244:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26873856, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd009 00010000:00010000:0.0:1789760780.888253:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b33 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.888269:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760780.888273:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.888314:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.888328:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760780.917149:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26873856 -> 26877952 00010000:00010000:2.0:1789760780.917167:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.917190:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b33 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.917199:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b33 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.917207:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760780.917211:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b33 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.917220:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b33 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.917239:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b33 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.917246:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547b33 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.917364:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495923200:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29167us (29849us total) trans 4294973878 rc 0/0 00000100:00100000:2.0:1789760780.917373:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6590 00000100:00100000:2.0:1789760780.917382:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765174315936 time 765208893709 state 0x2 ref 0 00000001:00080000:0.0:1789760780.918043:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973878 is committed 00000100:00100000:2.0:1789760780.922675:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760780.922771:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495923328 00000100:00100000:0.0:1789760780.922782:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495923328 00000100:00100000:0.0:1789760780.922835:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765174315936 time 765208893709 state 0x0 ref 1 00000100:00100000:0.0:1789760780.922851:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765209154125 time 765208893709 state 0x1 ref 1 00000100:00100000:0.0:1789760780.922879:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765209154125 time 765243596364 state 0x0 ref 1 00000100:00100000:0.0:1789760780.922892:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495923328:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760780.922936:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26877952, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd009 00010000:00010000:0.0:1789760780.922945:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547b3a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.922961:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760780.922966:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.922972:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547b3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.922986:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547b3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760780.962076:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26877952 -> 26882048 00010000:00010000:2.0:1789760780.962096:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547b3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.962109:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547b3a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.962118:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547b3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.962125:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760780.962130:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547b3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.962139:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547b3a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.962159:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547b3a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.962171:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547b3a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.962210:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973879 is committed 00000100:00100000:2.0:1789760780.962267:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495923328:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39372us (39595us total) trans 4294973879 rc 0/0 00000100:00100000:2.0:1789760780.962868:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6591 00000100:00100000:2.0:1789760780.962883:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765209154125 time 765243596364 state 0x2 ref 0 00000100:00100000:0.0:1789760780.966678:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760780.967684:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495923456 00000100:00100000:1.0:1789760780.967695:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495923456 00000100:00100000:1.0:1789760780.967763:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765209154125 time 765243596364 state 0x0 ref 1 00000100:00100000:1.0:1789760780.967855:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765243856780 time 765243596364 state 0x1 ref 1 00000100:00100000:1.0:1789760780.967933:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765243856780 time 765288648602 state 0x0 ref 1 00000100:00100000:1.0:1789760780.967947:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495923456:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760780.967994:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26882048, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd009 00010000:00010000:1.0:1789760780.968004:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547b41 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.968019:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760780.968024:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.968029:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547b41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.968043:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547b41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.999915:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973880 is committed 00002000:00010000:2.0:1789760781.000011:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26882048 -> 26886144 00010000:00010000:2.0:1789760781.000027:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547b41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.000041:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547b41 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.000049:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547b41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.000057:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760781.000061:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547b41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.000071:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547b41 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.000095:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547b41 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.000103:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547b41 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.000194:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495923456:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32246us (33522us total) trans 4294973880 rc 0/0 00000100:00100000:2.0:1789760781.000205:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6592 00000100:00100000:2.0:1789760781.000215:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765243856780 time 765288648602 state 0x2 ref 0 00000100:00100000:3.0:1789760781.003053:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760781.003154:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495923584 00000100:00100000:2.0:1789760781.003166:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495923584 00000100:00100000:2.0:1789760781.003241:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765243856780 time 765288648602 state 0x0 ref 1 00000100:00100000:2.0:1789760781.003256:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765288909018 time 765288648602 state 0x1 ref 1 00000100:00100000:2.0:1789760781.003321:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765288909018 time 765324036837 state 0x0 ref 1 00000100:00100000:2.0:1789760781.003334:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495923584:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760781.003380:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26886144, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd009 00010000:00010000:2.0:1789760781.003389:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547b48 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.003406:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760781.003412:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.003419:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547b48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.003433:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547b48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760781.004093:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789760781.034097:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26886144 -> 26890240 00010000:00010000:3.0:1789760781.034117:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547b48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.034131:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547b48 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.034140:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547b48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.034148:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760781.034152:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547b48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.034162:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547b48 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.034186:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547b48 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.034194:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547b48 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.034330:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495923584:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30984us (31271us total) trans 4294973881 rc 0/0 00000100:00100000:3.0:1789760781.034343:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6593 00000100:00100000:3.0:1789760781.034354:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765288909018 time 765324036837 state 0x2 ref 0 00000001:00080000:3.0:1789760781.034400:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973881 is committed 00000100:00100000:3.0:1789760781.042782:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760781.042928:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495923712 00000100:00100000:2.0:1789760781.042937:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495923712 00000100:00100000:2.0:1789760781.042988:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765288909018 time 765324036837 state 0x0 ref 1 00000100:00100000:2.0:1789760781.043003:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765324297253 time 765324036837 state 0x1 ref 1 00000100:00100000:2.0:1789760781.043032:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765324297253 time 765363749094 state 0x0 ref 1 00000100:00100000:2.0:1789760781.043044:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495923712:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760781.043090:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26890240, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd009 00010000:00010000:2.0:1789760781.043098:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547b4f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.043113:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760781.043117:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.043123:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547b4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.043137:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547b4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.080550:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973882 is committed 00002000:00010000:2.0:1789760781.080641:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26890240 -> 26894336 00010000:00010000:2.0:1789760781.080656:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547b4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.080668:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547b4f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.080678:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547b4f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.080685:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760781.080691:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547b4f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.080700:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547b4f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.080718:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547b4f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.080743:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547b4f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.080844:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495923712:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37796us (38066us total) trans 4294973882 rc 0/0 00000100:00100000:2.0:1789760781.080858:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6594 00000100:00100000:2.0:1789760781.080868:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765324297253 time 765363749094 state 0x2 ref 0 00000100:00100000:1.0:1789760781.088451:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760781.088687:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495923840 00000100:00100000:2.0:1789760781.088697:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495923840 00000100:00100000:2.0:1789760781.088761:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765324297253 time 765363749094 state 0x0 ref 1 00000100:00100000:2.0:1789760781.088777:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765364009510 time 765363749094 state 0x1 ref 1 00000100:00100000:2.0:1789760781.088808:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765364009510 time 765409524298 state 0x0 ref 1 00000100:00100000:2.0:1789760781.088820:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495923840:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760781.088874:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26894336, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd009 00010000:00010000:2.0:1789760781.088884:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547b56 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.088901:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760781.088905:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.088911:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547b56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.088926:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547b56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.128262:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973883 is committed 00002000:00010000:1.0:1789760781.128398:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26894336 -> 26898432 00010000:00010000:1.0:1789760781.128413:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547b56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.128426:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547b56 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.128441:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547b56 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.128448:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760781.128453:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547b56 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.128462:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547b56 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.128481:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547b56 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.128489:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354547b56 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.128580:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495923840:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39758us (40134us total) trans 4294973883 rc 0/0 00000100:00100000:1.0:1789760781.128591:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6595 00000100:00100000:1.0:1789760781.128601:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765364009510 time 765409524298 state 0x2 ref 0 00000100:00100000:3.0:1789760781.131989:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760781.132096:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495923968 00000100:00100000:2.0:1789760781.132106:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495923968 00000100:00100000:2.0:1789760781.132157:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765364009510 time 765409524298 state 0x0 ref 1 00000100:00100000:2.0:1789760781.132169:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765409784714 time 765409524298 state 0x1 ref 1 00000100:00100000:2.0:1789760781.132194:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765409784714 time 765452911156 state 0x0 ref 1 00000100:00100000:2.0:1789760781.132202:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495923968:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760781.132240:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26898432, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd009 00010000:00010000:2.0:1789760781.132246:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547b5d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.132258:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760781.132261:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.132266:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547b5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.132275:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547b5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.151210:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760781.159417:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973884 is committed 00002000:00010000:0.0:1789760781.159589:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26898432 -> 26902528 00010000:00010000:0.0:1789760781.159604:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547b5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.159616:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547b5d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.159625:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547b5d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.159633:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760781.159637:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547b5d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.159646:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547b5d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.159666:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547b5d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.159674:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547b5d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.159786:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495923968:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27578us (27799us total) trans 4294973884 rc 0/0 00000100:00100000:0.0:1789760781.159796:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6596 00000100:00100000:0.0:1789760781.159805:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765409784714 time 765452911156 state 0x2 ref 0 00000100:00100000:3.0:1789760781.165664:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760781.165733:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495924096 00000100:00100000:2.0:1789760781.165753:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495924096 00000100:00100000:2.0:1789760781.165811:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765409784714 time 765452911156 state 0x0 ref 1 00000100:00100000:2.0:1789760781.165827:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765453171572 time 765452911156 state 0x1 ref 1 00000100:00100000:2.0:1789760781.165864:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765453171572 time 765486580277 state 0x0 ref 1 00000100:00100000:2.0:1789760781.165877:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495924096:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760781.165924:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26902528, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd009 00010000:00010000:2.0:1789760781.165932:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b64 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.165947:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760781.165951:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.165956:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.165969:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760781.191218:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26902528 -> 26906624 00010000:00010000:2.0:1789760781.191238:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.191252:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b64 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.191261:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.191269:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760781.191273:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.191311:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b64 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.191331:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b64 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.191339:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b64 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.191420:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495924096:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25541us (25762us total) trans 4294973885 rc 0/0 00000100:00100000:2.0:1789760781.191430:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6597 00000100:00100000:2.0:1789760781.191441:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765453171572 time 765486580277 state 0x2 ref 0 00000001:00080000:3.0:1789760781.191529:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973885 is committed 00000100:00100000:2.0:1789760781.196210:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760781.196338:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495924224 00000100:00100000:1.0:1789760781.196349:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495924224 00000100:00100000:1.0:1789760781.196405:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765453171572 time 765486580277 state 0x0 ref 1 00000100:00100000:1.0:1789760781.196420:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765486840693 time 765486580277 state 0x1 ref 1 00000100:00100000:1.0:1789760781.196451:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765486840693 time 765517167775 state 0x0 ref 1 00000100:00100000:1.0:1789760781.196463:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495924224:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760781.196513:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26906624, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd009 00010000:00010000:1.0:1789760781.196523:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354547b6b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.196538:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760781.196543:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.196550:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354547b6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.196564:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354547b6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.216422:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973886 is committed 00002000:00010000:0.0:1789760781.216739:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26906624 -> 26910720 00010000:00010000:0.0:1789760781.216765:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354547b6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.216910:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354547b6b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.216920:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354547b6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.216927:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760781.216931:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354547b6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.216946:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354547b6b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.217266:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354547b6b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.217275:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354547b6b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.217390:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495924224:12345-192.168.203.9@tcp:4:dd.0 Request processed in 20924us (21184us total) trans 4294973886 rc 0/0 00000100:00100000:0.0:1789760781.217399:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6598 00000100:00100000:0.0:1789760781.217418:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765486840693 time 765517167775 state 0x2 ref 0 00000100:00100000:1.0:1789760781.223973:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760781.224410:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495924352 00000100:00100000:0.0:1789760781.224421:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495924352 00000100:00100000:0.0:1789760781.224474:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765486840693 time 765517167775 state 0x0 ref 1 00000100:00100000:0.0:1789760781.224489:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765517428191 time 765517167775 state 0x1 ref 1 00000100:00100000:0.0:1789760781.224525:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765517428191 time 765545241687 state 0x0 ref 1 00000100:00100000:0.0:1789760781.224538:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495924352:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760781.224589:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26910720, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd009 00010000:00010000:0.0:1789760781.224598:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547b72 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.224614:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760781.224618:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.224624:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547b72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.224635:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547b72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.255077:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973887 is committed 00002000:00010000:1.0:1789760781.255242:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26910720 -> 26914816 00010000:00010000:1.0:1789760781.255258:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547b72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.255270:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547b72 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.255296:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547b72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.255385:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760781.255392:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547b72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.255402:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547b72 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.255421:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547b72 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.255429:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547b72 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.255568:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495924352:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31026us (31598us total) trans 4294973887 rc 0/0 00000100:00100000:1.0:1789760781.255580:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6599 00000100:00100000:1.0:1789760781.255591:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765517428191 time 765545241687 state 0x2 ref 0 00000100:00100000:3.0:1789760781.262884:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760781.263159:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495924480 00000100:00100000:2.0:1789760781.263169:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495924480 00000100:00100000:2.0:1789760781.263225:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765517428191 time 765545241687 state 0x0 ref 1 00000100:00100000:2.0:1789760781.263240:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765545502103 time 765545241687 state 0x1 ref 1 00000100:00100000:2.0:1789760781.263268:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765545502103 time 765583984948 state 0x0 ref 1 00000100:00100000:2.0:1789760781.263490:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495924480:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760781.263541:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26914816, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd009 00010000:00010000:2.0:1789760781.263550:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b79 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.263571:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760781.263576:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.263582:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.263595:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.301364:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973888 is committed 00002000:00010000:2.0:1789760781.301461:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26914816 -> 26918912 00010000:00010000:2.0:1789760781.301476:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.301489:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b79 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.301499:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b79 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.301506:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760781.301511:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b79 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.301520:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b79 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.301540:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b79 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.301550:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b79 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.301785:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495924480:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38500us (38903us total) trans 4294973888 rc 0/0 00000100:00100000:2.0:1789760781.301796:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6600 00000100:00100000:2.0:1789760781.301806:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765545502103 time 765583984948 state 0x2 ref 0 00000100:00100000:3.0:1789760781.305406:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760781.305472:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495924608 00000100:00100000:0.0:1789760781.305482:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495924608 00000100:00100000:0.0:1789760781.305610:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765545502103 time 765583984948 state 0x0 ref 1 00000100:00100000:0.0:1789760781.305698:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765584245364 time 765583984948 state 0x1 ref 1 00000100:00100000:0.0:1789760781.305729:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765584245364 time 765626445273 state 0x0 ref 1 00000100:00100000:0.0:1789760781.305752:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495924608:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760781.305801:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26918912, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd009 00010000:00010000:0.0:1789760781.305810:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547b80 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.305825:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760781.305830:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.305836:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547b80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.305854:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547b80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760781.332771:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26918912 -> 26923008 00010000:00010000:1.0:1789760781.332791:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547b80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.332803:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547b80 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.332818:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547b80 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.332825:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760781.332830:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547b80 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.332838:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547b80 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.332858:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547b80 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.332867:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547b80 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.333081:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495924608:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27337us (27676us total) trans 4294973889 rc 0/0 00000100:00100000:1.0:1789760781.333632:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6601 00000100:00100000:1.0:1789760781.333642:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765584245364 time 765626445273 state 0x2 ref 0 00000001:00080000:1.0:1789760781.334618:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973889 is committed 00000100:00100000:1.0:1789760781.339198:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760781.339372:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495924736 00000100:00100000:0.0:1789760781.339382:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495924736 00000100:00100000:0.0:1789760781.339438:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765584245364 time 765626445273 state 0x0 ref 1 00000100:00100000:0.0:1789760781.339452:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765626705689 time 765626445273 state 0x1 ref 1 00000100:00100000:3.0:1789760781.339580:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765626705689 time 765660293769 state 0x0 ref 1 00000100:00100000:3.0:1789760781.339596:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034ce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495924736:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760781.339650:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26923008, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd009 00010000:00010000:3.0:1789760781.339659:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547b87 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.339675:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:3.0:1789760781.339680:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.339688:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547b87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.339706:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547b87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.378946:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973890 is committed 00002000:00010000:1.0:1789760781.379101:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26923008 -> 26927104 00010000:00010000:1.0:1789760781.379117:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547b87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.379130:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547b87 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.379139:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547b87 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.379146:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760781.379151:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547b87 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.379161:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547b87 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.379191:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547b87 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.379199:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547b87 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.379319:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034ce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495924736:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39687us (40090us total) trans 4294973890 rc 0/0 00000100:00100000:1.0:1789760781.379330:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6602 00000100:00100000:1.0:1789760781.379340:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765626705689 time 765660293769 state 0x2 ref 0 00000100:00100000:3.0:1789760781.383224:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760781.383558:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495924864 00000100:00100000:2.0:1789760781.383569:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495924864 00000100:00100000:2.0:1789760781.383623:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765626705689 time 765660293769 state 0x0 ref 1 00000100:00100000:2.0:1789760781.383637:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765660554185 time 765660293769 state 0x1 ref 1 00000100:00100000:2.0:1789760781.383713:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765660554185 time 765704428733 state 0x0 ref 1 00000100:00100000:2.0:1789760781.383727:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495924864:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760781.383784:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26927104, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd009 00010000:00010000:2.0:1789760781.383793:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b8e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.383808:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760781.383813:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.383825:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.383844:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.411977:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973891 is committed 00002000:00010000:2.0:1789760781.412080:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26927104 -> 26931200 00010000:00010000:2.0:1789760781.412095:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.412109:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b8e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.412117:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.412125:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760781.412130:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.412140:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b8e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.412161:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b8e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.412170:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547b8e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.412366:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495924864:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28637us (29145us total) trans 4294973891 rc 0/0 00000100:00100000:2.0:1789760781.412377:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6603 00000100:00100000:2.0:1789760781.412387:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765660554185 time 765704428733 state 0x2 ref 0 00000100:00100000:3.0:1789760781.432576:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760781.432698:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495924992 00000100:00100000:1.0:1789760781.432709:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495924992 00000100:00100000:1.0:1789760781.432827:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765660554185 time 765704428733 state 0x0 ref 1 00000100:00100000:1.0:1789760781.432842:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765704689149 time 765704428733 state 0x1 ref 1 00000100:00100000:1.0:1789760781.432942:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765704689149 time 765753658056 state 0x0 ref 1 00000100:00100000:1.0:1789760781.432956:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495924992:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760781.433007:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26931200, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd009 00010000:00010000:1.0:1789760781.433016:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354547b95 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.433034:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760781.433039:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.433045:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354547b95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.433058:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354547b95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.478512:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973892 is committed 00002000:00010000:1.0:1789760781.479919:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26931200 -> 26935296 00010000:00010000:1.0:1789760781.479937:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354547b95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.479950:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354547b95 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.479966:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354547b95 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.479975:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760781.479981:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354547b95 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.479989:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354547b95 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.480009:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354547b95 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.480017:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354547b95 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.480116:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495924992:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47158us (47544us total) trans 4294973892 rc 0/0 00000100:00100000:1.0:1789760781.480127:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6604 00000100:00100000:1.0:1789760781.480138:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765704689149 time 765753658056 state 0x2 ref 0 00000100:00100000:1.0:1789760781.484978:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760781.485152:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495925120 00000100:00100000:3.0:1789760781.485162:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495925120 00000100:00100000:3.0:1789760781.485209:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765704689149 time 765753658056 state 0x0 ref 1 00000100:00100000:3.0:1789760781.485223:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765753918472 time 765753658056 state 0x1 ref 1 00000100:00100000:3.0:1789760781.485311:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765753918472 time 765806027541 state 0x0 ref 1 00000100:00100000:3.0:1789760781.485323:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495925120:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760781.485366:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26935296, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd009 00010000:00010000:3.0:1789760781.485375:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547b9c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.485396:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:3.0:1789760781.485400:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.485406:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547b9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.485419:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547b9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.517509:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973893 is committed 00002000:00010000:1.0:1789760781.517553:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26935296 -> 26939392 00010000:00010000:1.0:1789760781.517570:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547b9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.517591:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547b9c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.517599:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547b9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.517603:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760781.517607:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547b9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.517613:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547b9c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.517631:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547b9c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.517636:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547b9c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.518104:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495925120:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32399us (32752us total) trans 4294973893 rc 0/0 00000100:00100000:1.0:1789760781.518115:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6605 00000100:00100000:1.0:1789760781.518124:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765753918472 time 765806027541 state 0x2 ref 0 00000100:00100000:3.0:1789760781.521569:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760781.522571:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495925248 00000100:00100000:2.0:1789760781.522583:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495925248 00000100:00100000:2.0:1789760781.522641:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765753918472 time 765806027541 state 0x0 ref 1 00000100:00100000:2.0:1789760781.522657:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765806287957 time 765806027541 state 0x1 ref 1 00000100:00100000:2.0:1789760781.522690:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765806287957 time 765843407084 state 0x0 ref 1 00000100:00100000:2.0:1789760781.522702:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495925248:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760781.523430:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26939392, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd009 00010000:00010000:2.0:1789760781.523440:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ba3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.523456:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760781.523460:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.523466:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ba3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.523479:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ba3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.583061:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973894 is committed 00002000:00010000:2.0:1789760781.583175:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26939392 -> 26943488 00010000:00010000:2.0:1789760781.583192:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ba3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.583206:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ba3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.583215:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ba3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.583224:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760781.583230:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ba3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.583239:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ba3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.583259:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ba3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.583268:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ba3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.583428:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495925248:12345-192.168.203.9@tcp:4:dd.0 Request processed in 60722us (61863us total) trans 4294973894 rc 0/0 00000100:00100000:2.0:1789760781.583442:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6606 00000100:00100000:2.0:1789760781.583454:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765806287957 time 765843407084 state 0x2 ref 0 00000100:00100000:2.0:1789760781.587357:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760781.587466:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495925376 00000100:00100000:1.0:1789760781.587475:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495925376 00000100:00100000:1.0:1789760781.587530:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765806287957 time 765843407084 state 0x0 ref 1 00000100:00100000:1.0:1789760781.587547:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765843667500 time 765843407084 state 0x1 ref 1 00000100:00100000:1.0:1789760781.587578:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765843667500 time 765908294600 state 0x0 ref 1 00000100:00100000:1.0:1789760781.587590:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495925376:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760781.587636:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26943488, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd009 00010000:00010000:1.0:1789760781.587646:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547baa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.587661:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760781.587666:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.587672:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547baa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.587685:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547baa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.598627:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f034cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520315136:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760781.598700:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760781.598796:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520315136 00000100:00100000:1.0:1789760781.598802:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520315136 00000100:00100000:1.0:1789760781.598850:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 215 00000100:00100000:1.0:1789760781.598858:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520315136:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760781.598880:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527892480 pending 1703936 free 3831496704 avail 3807379456 00000100:00100000:1.0:1789760781.598922:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520315136:12345-0@lo:13:osp-pre-0-0.0 Request processed in 64us (224us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760781.598930:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 215 00000100:00080000:1.0:1789760781.598950:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760781.598983:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f034cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520315136:0@lo:13:osp-pre-0-0.0 00000001:00080000:1.0:1789760781.614539:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973895 is committed 00002000:00010000:2.0:1789760781.614867:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26943488 -> 26947584 00010000:00010000:2.0:1789760781.614884:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547baa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.614897:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547baa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.614906:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547baa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.614915:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760781.614921:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547baa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.614931:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547baa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.614950:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547baa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.614958:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354547baa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.615056:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495925376:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27464us (27782us total) trans 4294973895 rc 0/0 00000100:00100000:2.0:1789760781.615068:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6607 00000100:00100000:2.0:1789760781.615078:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765843667500 time 765908294600 state 0x2 ref 0 00000100:00100000:1.0:1789760781.618461:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760781.618548:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495925504 00000100:00100000:3.0:1789760781.618557:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495925504 00000100:00100000:3.0:1789760781.618611:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765843667500 time 765908294600 state 0x0 ref 1 00000100:00100000:3.0:1789760781.618626:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765908555016 time 765908294600 state 0x1 ref 1 00000100:00100000:3.0:1789760781.618660:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765908555016 time 765939376394 state 0x0 ref 1 00000100:00100000:3.0:1789760781.618670:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495925504:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760781.618707:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26947584, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd009 00010000:00010000:3.0:1789760781.618713:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547bb1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.618732:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:3.0:1789760781.618736:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.618751:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547bb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.618884:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547bb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.654270:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973896 is committed 00002000:00010000:2.0:1789760781.654427:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26947584 -> 26951680 00002000:00010000:2.0:1789760781.654431:0:6714:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760780 -> 1789760781 00002000:00010000:2.0:1789760781.654434:0:6714:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760780 -> 1789760781 00010000:00010000:2.0:1789760781.654450:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547bb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.654462:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547bb1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.654471:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547bb1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.654478:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760781.654482:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547bb1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.654491:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547bb1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.654509:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547bb1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.654517:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547bb1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.654605:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495925504:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35932us (36148us total) trans 4294973896 rc 0/0 00000100:00100000:2.0:1789760781.654613:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6608 00000100:00100000:2.0:1789760781.654623:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765908555016 time 765939376394 state 0x2 ref 0 00000100:00100000:3.0:1789760781.662522:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760781.662610:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495925632 00000100:00100000:2.0:1789760781.662620:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495925632 00000100:00100000:2.0:1789760781.662672:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765908555016 time 765939376394 state 0x0 ref 1 00000100:00100000:2.0:1789760781.662686:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765939636810 time 765939376394 state 0x1 ref 1 00000100:00100000:2.0:1789760781.662714:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765939636810 time 765983431584 state 0x0 ref 1 00000100:00100000:2.0:1789760781.662736:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495925632:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760781.662915:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26951680, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd009 00010000:00010000:2.0:1789760781.662925:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547bb8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.662942:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760781.662946:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.662953:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547bb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.662973:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547bb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760781.697505:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26951680 -> 26955776 00010000:00010000:2.0:1789760781.697526:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547bb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.697540:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547bb8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.697550:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547bb8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.697558:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760781.697564:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547bb8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.697574:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547bb8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.697595:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547bb8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.697618:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547bb8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.697705:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495925632:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34966us (35189us total) trans 4294973897 rc 0/0 00000100:00100000:2.0:1789760781.697716:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6609 00000001:00080000:0.0:1789760781.698208:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973897 is committed 00000100:00100000:2.0:1789760781.700990:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765939636810 time 765983431584 state 0x2 ref 0 00000100:00100000:3.0:1789760781.702484:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760781.702677:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495925760 00000100:00100000:2.0:1789760781.702686:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495925760 00000100:00100000:2.0:1789760781.702800:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765939636810 time 765983431584 state 0x0 ref 1 00000100:00100000:2.0:1789760781.702880:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765983692000 time 765983431584 state 0x1 ref 1 00000100:00100000:2.0:1789760781.702937:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765983692000 time 766023652843 state 0x0 ref 1 00000100:00100000:2.0:1789760781.702951:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495925760:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760781.702998:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26955776, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd009 00010000:00010000:2.0:1789760781.703007:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bbf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.703024:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760781.703029:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.703036:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bbf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.703048:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bbf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.733544:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973898 is committed 00002000:00010000:2.0:1789760781.733650:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26955776 -> 26959872 00010000:00010000:2.0:1789760781.733667:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bbf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.733680:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bbf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.733689:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bbf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.733697:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760781.733702:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bbf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.733712:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bbf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.733732:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bbf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.733749:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bbf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.733835:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495925760:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30882us (31354us total) trans 4294973898 rc 0/0 00000100:00100000:2.0:1789760781.733847:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6610 00000100:00100000:2.0:1789760781.733857:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765983692000 time 766023652843 state 0x2 ref 0 00000100:00100000:1.0:1789760781.739039:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760781.739137:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495925888 00000100:00100000:2.0:1789760781.739148:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495925888 00000100:00100000:2.0:1789760781.739438:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 765983692000 time 766023652843 state 0x0 ref 1 00000100:00100000:2.0:1789760781.739464:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766023913259 time 766023652843 state 0x1 ref 1 00000100:00100000:2.0:1789760781.739495:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766023913259 time 766060211202 state 0x0 ref 1 00000100:00100000:2.0:1789760781.739507:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495925888:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760781.739674:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26959872, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd009 00010000:00010000:2.0:1789760781.739684:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bc6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.739700:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760781.739705:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.739720:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bc6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.739734:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bc6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760781.762654:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26959872 -> 26963968 00010000:00010000:2.0:1789760781.762693:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bc6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.762721:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bc6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.762730:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973899 is committed 00010000:00010000:2.0:1789760781.762732:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bc6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.762751:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760781.762756:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bc6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.762766:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bc6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.762787:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bc6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.762795:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bc6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.762884:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495925888:12345-192.168.203.9@tcp:4:dd.0 Request processed in 23375us (23850us total) trans 4294973899 rc 0/0 00000100:00100000:2.0:1789760781.762895:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6611 00000100:00100000:2.0:1789760781.762906:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766023913259 time 766060211202 state 0x2 ref 0 00000100:00100000:3.0:1789760781.770204:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760781.770304:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495926016 00000100:00100000:2.0:1789760781.770314:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495926016 00000100:00100000:2.0:1789760781.770365:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766023913259 time 766060211202 state 0x0 ref 1 00000100:00100000:2.0:1789760781.770379:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766060471618 time 766060211202 state 0x1 ref 1 00000100:00100000:2.0:1789760781.770411:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766060471618 time 766091128495 state 0x0 ref 1 00000100:00100000:2.0:1789760781.770425:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495926016:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760781.770473:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26963968, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd009 00010000:00010000:2.0:1789760781.770485:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bcd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.770501:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760781.770507:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.770514:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bcd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.770531:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bcd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.791443:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973900 is committed 00002000:00010000:2.0:1789760781.791658:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26963968 -> 26968064 00010000:00010000:2.0:1789760781.791674:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bcd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.791686:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bcd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.791701:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bcd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.791708:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760781.791713:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bcd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.791732:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bcd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.791759:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bcd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.791768:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bcd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.791890:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495926016:12345-192.168.203.9@tcp:4:dd.0 Request processed in 21462us (21689us total) trans 4294973900 rc 0/0 00000100:00100000:2.0:1789760781.791901:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6612 00000100:00100000:2.0:1789760781.791911:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766060471618 time 766091128495 state 0x2 ref 0 00000100:00100000:1.0:1789760781.797447:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760781.797545:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495926144 00000100:00100000:2.0:1789760781.797557:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495926144 00000100:00100000:2.0:1789760781.797614:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766060471618 time 766091128495 state 0x0 ref 1 00000100:00100000:2.0:1789760781.797629:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766091388911 time 766091128495 state 0x1 ref 1 00000100:00100000:2.0:1789760781.797708:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766091388911 time 766118423561 state 0x0 ref 1 00000100:00100000:2.0:1789760781.797723:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495926144:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760781.797792:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26968064, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd009 00010000:00010000:2.0:1789760781.797807:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bd4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.797823:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760781.797828:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.797834:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bd4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.797847:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bd4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760781.829995:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26968064 -> 26972160 00010000:00010000:2.0:1789760781.830015:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bd4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.830029:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bd4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.830038:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bd4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.830046:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760781.830052:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bd4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.830061:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bd4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.830079:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bd4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.830087:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bd4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.830179:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495926144:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32454us (32736us total) trans 4294973901 rc 0/0 00000100:00100000:2.0:1789760781.830189:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6613 00000100:00100000:2.0:1789760781.830199:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766091388911 time 766118423561 state 0x2 ref 0 00000001:00080000:0.0:1789760781.830945:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973901 is committed 00000100:00100000:1.0:1789760781.836671:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760781.836768:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495926272 00000100:00100000:2.0:1789760781.836778:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495926272 00000100:00100000:2.0:1789760781.836834:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766091388911 time 766118423561 state 0x0 ref 1 00000100:00100000:2.0:1789760781.836849:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766118683977 time 766118423561 state 0x1 ref 1 00000100:00100000:2.0:1789760781.837173:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766118683977 time 766157887699 state 0x0 ref 1 00000100:00100000:2.0:1789760781.837188:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495926272:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760781.837239:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26972160, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd009 00010000:00010000:2.0:1789760781.837248:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bdb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.837264:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760781.837269:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.837275:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bdb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.837327:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bdb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.870574:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973902 is committed 00002000:00010000:2.0:1789760781.870658:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26972160 -> 26976256 00010000:00010000:2.0:1789760781.870673:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bdb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.870685:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bdb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.870694:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bdb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.870702:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760781.870707:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bdb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.870716:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bdb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.870813:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bdb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.870820:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547bdb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.870944:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495926272:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33754us (34277us total) trans 4294973902 rc 0/0 00000100:00100000:2.0:1789760781.870954:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6614 00000100:00100000:2.0:1789760781.870964:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766118683977 time 766157887699 state 0x2 ref 0 00000100:00100000:1.0:1789760781.874264:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760781.874412:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495926400 00000100:00100000:2.0:1789760781.874420:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495926400 00000100:00100000:2.0:1789760781.874464:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766118683977 time 766157887699 state 0x0 ref 1 00000100:00100000:2.0:1789760781.874475:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766158148115 time 766157887699 state 0x1 ref 1 00000100:00100000:2.0:1789760781.874496:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766158148115 time 766195213002 state 0x0 ref 1 00000100:00100000:2.0:1789760781.874508:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495926400:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760781.874554:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26976256, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd009 00010000:00010000:2.0:1789760781.874563:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547be2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.874578:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760781.874583:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.874589:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547be2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.874602:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547be2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.896691:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973903 is committed 00002000:00010000:0.0:1789760781.896873:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26976256 -> 26980352 00010000:00010000:0.0:1789760781.896889:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547be2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.896903:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547be2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.896912:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547be2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.896919:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760781.896923:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547be2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.896936:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547be2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.896955:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547be2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.896963:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547be2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.897050:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495926400:12345-192.168.203.9@tcp:4:dd.0 Request processed in 22540us (22788us total) trans 4294973903 rc 0/0 00000100:00100000:0.0:1789760781.897058:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6615 00000100:00100000:0.0:1789760781.897066:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766158148115 time 766195213002 state 0x2 ref 0 00000100:00100000:1.0:1789760781.900221:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760781.900335:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495926528 00000100:00100000:2.0:1789760781.900351:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495926528 00000100:00100000:2.0:1789760781.900419:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766158148115 time 766195213002 state 0x0 ref 1 00000100:00100000:2.0:1789760781.900435:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766195473418 time 766195213002 state 0x1 ref 1 00000100:00100000:2.0:1789760781.900463:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766195473418 time 766221179666 state 0x0 ref 1 00000100:00100000:2.0:1789760781.900486:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495926528:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760781.900538:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26980352, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd009 00010000:00010000:2.0:1789760781.900548:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547be9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.900625:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760781.900632:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.900639:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547be9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.900654:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547be9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.941167:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973904 is committed 00002000:00010000:2.0:1789760781.941275:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26980352 -> 26984448 00010000:00010000:2.0:1789760781.941309:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547be9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.941321:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547be9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.941330:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547be9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.941337:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760781.941342:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547be9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.941352:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547be9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.941370:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547be9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.941377:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547be9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.941474:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495926528:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40995us (41256us total) trans 4294973904 rc 0/0 00000100:00100000:2.0:1789760781.941488:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6616 00000100:00100000:2.0:1789760781.941498:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766195473418 time 766221179666 state 0x2 ref 0 00000100:00100000:2.0:1789760781.945359:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760781.945446:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495926656 00000100:00100000:0.0:1789760781.945457:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495926656 00000100:00100000:0.0:1789760781.945512:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766195473418 time 766221179666 state 0x0 ref 1 00000100:00100000:0.0:1789760781.945528:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766221440082 time 766221179666 state 0x1 ref 1 00000100:00100000:0.0:1789760781.945558:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766221440082 time 766266275075 state 0x0 ref 1 00000100:00100000:0.0:1789760781.945581:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495926656:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760781.945632:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26984448, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd009 00010000:00010000:0.0:1789760781.945642:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547bf0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.945659:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:0.0:1789760781.945663:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.945670:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547bf0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.945684:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547bf0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.983544:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973905 is committed 00002000:00010000:2.0:1789760781.983726:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26984448 -> 26988544 00010000:00010000:2.0:1789760781.983752:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547bf0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.983830:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547bf0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.983840:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547bf0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.983847:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760781.983851:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547bf0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.983860:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547bf0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.983880:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547bf0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.983888:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547bf0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.983989:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495926656:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38405us (38636us total) trans 4294973905 rc 0/0 00000100:00100000:2.0:1789760781.984000:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6617 00000100:00100000:2.0:1789760781.984011:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766221440082 time 766266275075 state 0x2 ref 0 00000100:00100000:3.0:1789760781.992393:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760781.992940:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495926784 00000100:00100000:0.0:1789760781.992950:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495926784 00000100:00100000:0.0:1789760781.993005:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766221440082 time 766266275075 state 0x0 ref 1 00000100:00100000:0.0:1789760781.993022:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766266535491 time 766266275075 state 0x1 ref 1 00000100:00100000:0.0:1789760781.993111:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766266535491 time 766313826582 state 0x0 ref 1 00000100:00100000:0.0:1789760781.993123:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495926784:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760781.993170:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26988544, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd009 00010000:00010000:0.0:1789760781.993179:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547bf7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.993195:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:0.0:1789760781.993199:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.993205:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547bf7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.993219:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547bf7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.026316:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973906 is committed 00002000:00010000:2.0:1789760782.026467:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26988544 -> 26992640 00010000:00010000:2.0:1789760782.026488:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547bf7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.026502:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547bf7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.026511:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547bf7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.026519:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ae00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760782.026523:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547bf7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.026533:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547bf7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.026552:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547bf7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.026566:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547bf7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.026657:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495926784:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33532us (34270us total) trans 4294973906 rc 0/0 00000100:00100000:2.0:1789760782.026668:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6618 00000100:00100000:2.0:1789760782.026678:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766266535491 time 766313826582 state 0x2 ref 0 00000100:00100000:0.0:1789760782.032890:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760782.032980:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495926912 00000100:00100000:2.0:1789760782.032990:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495926912 00000100:00100000:2.0:1789760782.033044:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766266535491 time 766313826582 state 0x0 ref 1 00000100:00100000:2.0:1789760782.033059:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766314086998 time 766313826582 state 0x1 ref 1 00000100:00100000:2.0:1789760782.033086:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766314086998 time 766353803371 state 0x0 ref 1 00000100:00100000:2.0:1789760782.033098:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495926912:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760782.033143:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26992640, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd009 00010000:00010000:2.0:1789760782.033152:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547bfe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.033168:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760782.033172:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.033179:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547bfe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.033197:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547bfe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.093185:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973907 is committed 00002000:00010000:2.0:1789760782.093374:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26992640 -> 26996736 00010000:00010000:2.0:1789760782.093452:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547bfe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.093531:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547bfe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.093541:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547bfe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.093548:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760782.093553:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547bfe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.093561:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547bfe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.093581:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547bfe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.093589:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547bfe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.093685:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495926912:12345-192.168.203.9@tcp:4:dd.0 Request processed in 60584us (60801us total) trans 4294973907 rc 0/0 00000100:00100000:2.0:1789760782.093695:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6619 00000100:00100000:2.0:1789760782.093705:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766314086998 time 766353803371 state 0x2 ref 0 00000100:00100000:0.0:1789760782.107193:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760782.107264:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495927040 00000100:00100000:2.0:1789760782.107274:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495927040 00000100:00100000:2.0:1789760782.107354:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766314086998 time 766353803371 state 0x0 ref 1 00000100:00100000:2.0:1789760782.107369:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766354063787 time 766353803371 state 0x1 ref 1 00000100:00100000:2.0:1789760782.107404:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766354063787 time 766428121059 state 0x0 ref 1 00000100:00100000:2.0:1789760782.107417:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495927040:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760782.108015:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26996736, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd009 00010000:00010000:2.0:1789760782.108026:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547c05 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.108042:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760782.108046:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.108052:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547c05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.108116:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547c05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.147480:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973908 is committed 00002000:00010000:2.0:1789760782.147549:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26996736 -> 27000832 00010000:00010000:2.0:1789760782.147564:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547c05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.147577:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547c05 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.147587:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547c05 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.147594:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760782.147599:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547c05 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.147608:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547c05 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.147627:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547c05 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.147635:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547c05 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.147785:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495927040:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40299us (41003us total) trans 4294973908 rc 0/0 00000100:00100000:2.0:1789760782.147800:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6620 00000100:00100000:2.0:1789760782.147810:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766354063787 time 766428121059 state 0x2 ref 0 00000100:00100000:3.0:1789760782.150672:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760782.150770:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495927168 00000100:00100000:2.0:1789760782.150782:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495927168 00000100:00100000:2.0:1789760782.150839:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766354063787 time 766428121059 state 0x0 ref 1 00000100:00100000:2.0:1789760782.150855:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766428381475 time 766428121059 state 0x1 ref 1 00000100:00100000:2.0:1789760782.150884:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766428381475 time 766471601555 state 0x0 ref 1 00000100:00100000:2.0:1789760782.150897:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495927168:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760782.150948:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27000832, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd009 00010000:00010000:2.0:1789760782.150956:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c0c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.150972:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760782.150976:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.150983:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.150997:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.176950:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760782.187057:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27000832 -> 27004928 00010000:00010000:2.0:1789760782.187078:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.187090:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c0c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.187099:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c0c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.187106:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760782.187111:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c0c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.187121:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c0c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.187140:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c0c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.187147:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c0c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.187240:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495927168:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36340us (36571us total) trans 4294973909 rc 0/0 00000100:00100000:2.0:1789760782.187260:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6621 00000100:00100000:2.0:1789760782.187270:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766428381475 time 766471601555 state 0x2 ref 0 00000001:00080000:3.0:1789760782.187806:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973909 is committed 00000100:00100000:0.0:1789760782.192837:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760782.192969:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495927296 00000100:00100000:2.0:1789760782.192979:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495927296 00000100:00100000:2.0:1789760782.193031:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766428381475 time 766471601555 state 0x0 ref 1 00000100:00100000:2.0:1789760782.193046:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766471861971 time 766471601555 state 0x1 ref 1 00000100:00100000:2.0:1789760782.193083:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766471861971 time 766513799901 state 0x0 ref 1 00000100:00100000:2.0:1789760782.193096:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f872680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495927296:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760782.193140:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27004928, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd009 00010000:00010000:2.0:1789760782.193149:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c13 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.193165:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760782.193170:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.193176:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.193189:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.223564:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973910 is committed 00002000:00010000:3.0:1789760782.224506:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27004928 -> 27009024 00010000:00010000:3.0:1789760782.224522:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.224535:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c13 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.224544:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c13 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.224552:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760782.224557:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c13 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.224565:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c13 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.224586:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c13 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.224593:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c13 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.224801:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f872680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495927296:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31701us (31967us total) trans 4294973910 rc 0/0 00000100:00100000:3.0:1789760782.224812:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6622 00000100:00100000:3.0:1789760782.224822:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766471861971 time 766513799901 state 0x2 ref 0 00000100:00100000:1.0:1789760782.228886:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760782.229062:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495927424 00000100:00100000:2.0:1789760782.229073:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495927424 00000100:00100000:2.0:1789760782.229129:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766471861971 time 766513799901 state 0x0 ref 1 00000100:00100000:2.0:1789760782.229144:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766514060317 time 766513799901 state 0x1 ref 1 00000100:00100000:2.0:1789760782.229176:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766514060317 time 766549893476 state 0x0 ref 1 00000100:00100000:2.0:1789760782.229188:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03096680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495927424:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760782.229243:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27009024, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd009 00010000:00010000:2.0:1789760782.229491:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547c1a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.229508:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760782.229512:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.229518:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547c1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.229531:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547c1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760782.266750:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27009024 -> 27013120 00010000:00010000:3.0:1789760782.266771:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547c1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.266784:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547c1a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.266793:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547c1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.266800:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760782.266805:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547c1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.266819:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547c1a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.266839:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547c1a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.266845:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547c1a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.266936:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03096680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495927424:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37745us (38055us total) trans 4294973911 rc 0/0 00000100:00100000:3.0:1789760782.266946:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6623 00000100:00100000:3.0:1789760782.266955:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766514060317 time 766549893476 state 0x2 ref 0 00000001:00080000:1.0:1789760782.267086:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973911 is committed 00000100:00100000:1.0:1789760782.270905:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760782.271004:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495927552 00000100:00100000:2.0:1789760782.271013:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495927552 00000100:00100000:2.0:1789760782.271068:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766514060317 time 766549893476 state 0x0 ref 1 00000100:00100000:2.0:1789760782.271083:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766550153892 time 766549893476 state 0x1 ref 1 00000100:00100000:2.0:1789760782.271113:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766550153892 time 766591829521 state 0x0 ref 1 00000100:00100000:2.0:1789760782.271125:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495927552:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760782.271169:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27013120, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd009 00010000:00010000:2.0:1789760782.271178:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c21 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.271194:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760782.271198:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.271204:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.271218:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.307775:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973912 is committed 00002000:00010000:3.0:1789760782.307938:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27013120 -> 27017216 00010000:00010000:3.0:1789760782.307954:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.307967:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c21 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.307977:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c21 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.307985:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760782.307990:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c21 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.308000:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c21 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.308019:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c21 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.308026:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c21 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.308116:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495927552:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36989us (37215us total) trans 4294973912 rc 0/0 00000100:00100000:3.0:1789760782.308126:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6624 00000100:00100000:3.0:1789760782.308136:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766550153892 time 766591829521 state 0x2 ref 0 00000100:00100000:2.0:1789760782.311801:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760782.312199:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495927680 00000100:00100000:3.0:1789760782.312209:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495927680 00000100:00100000:3.0:1789760782.312265:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766550153892 time 766591829521 state 0x0 ref 1 00000100:00100000:3.0:1789760782.312325:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766592089937 time 766591829521 state 0x1 ref 1 00000100:00100000:3.0:1789760782.312347:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766592089937 time 766633064040 state 0x0 ref 1 00000100:00100000:3.0:1789760782.312358:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92901237f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495927680:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760782.312403:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27017216, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd009 00010000:00010000:3.0:1789760782.312412:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c28 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.312427:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760782.312432:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.312438:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.312451:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.343604:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973913 is committed 00002000:00010000:2.0:1789760782.343726:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27017216 -> 27021312 00010000:00010000:2.0:1789760782.343751:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.343765:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c28 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.343775:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c28 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.343783:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760782.343788:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c28 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.343802:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c28 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.343823:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c28 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.343831:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c28 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.344062:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92901237f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495927680:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31700us (32264us total) trans 4294973913 rc 0/0 00000100:00100000:2.0:1789760782.344076:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6625 00000100:00100000:2.0:1789760782.344086:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766592089937 time 766633064040 state 0x2 ref 0 00000100:00100000:2.0:1789760782.347996:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760782.348062:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495927808 00000100:00100000:3.0:1789760782.348073:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495927808 00000100:00100000:3.0:1789760782.348190:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766592089937 time 766633064040 state 0x0 ref 1 00000100:00100000:3.0:1789760782.348206:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766633324456 time 766633064040 state 0x1 ref 1 00000100:00100000:3.0:1789760782.348237:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766633324456 time 766668954663 state 0x0 ref 1 00000100:00100000:3.0:1789760782.348260:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495927808:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760782.348507:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27021312, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd009 00010000:00010000:3.0:1789760782.348514:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c2f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.348526:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760782.348529:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.348534:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.348548:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.368911:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973914 is committed 00002000:00010000:3.0:1789760782.368981:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27021312 -> 27025408 00010000:00010000:3.0:1789760782.368999:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.369012:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c2f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.369023:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c2f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.369031:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760782.369036:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c2f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.369046:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c2f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.369066:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c2f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.369075:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c2f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.369166:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495927808:12345-192.168.203.9@tcp:4:dd.0 Request processed in 20913us (21176us total) trans 4294973914 rc 0/0 00000100:00100000:3.0:1789760782.369177:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6626 00000100:00100000:3.0:1789760782.369187:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766633324456 time 766668954663 state 0x2 ref 0 00000100:00100000:1.0:1789760782.376671:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760782.376813:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495927936 00000100:00100000:3.0:1789760782.376823:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495927936 00000100:00100000:3.0:1789760782.376875:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766633324456 time 766668954663 state 0x0 ref 1 00000100:00100000:3.0:1789760782.376890:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766669215079 time 766668954663 state 0x1 ref 1 00000100:00100000:3.0:1789760782.376967:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766669215079 time 766697682269 state 0x0 ref 1 00000100:00100000:3.0:1789760782.376981:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495927936:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760782.377027:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27025408, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd009 00010000:00010000:3.0:1789760782.377036:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c36 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.377053:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760782.377057:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.377064:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.377078:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.420301:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973915 is committed 00002000:00010000:1.0:1789760782.420386:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27025408 -> 27029504 00010000:00010000:1.0:1789760782.420400:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.420414:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c36 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.420424:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c36 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.420431:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760782.420436:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c36 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.420446:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c36 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.420532:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c36 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.420540:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c36 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.423112:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495927936:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46121us (46438us total) trans 4294973915 rc 0/0 00000100:00100000:1.0:1789760782.423129:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6627 00000100:00100000:1.0:1789760782.423140:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766669215079 time 766697682269 state 0x2 ref 0 00000100:00100000:2.0:1789760782.435009:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760782.436103:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495928064 00000100:00100000:3.0:1789760782.436116:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495928064 00000100:00100000:3.0:1789760782.436175:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766669215079 time 766697682269 state 0x0 ref 1 00000100:00100000:3.0:1789760782.436191:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766697942685 time 766697682269 state 0x1 ref 1 00000100:00100000:3.0:1789760782.436224:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766697942685 time 766756941594 state 0x0 ref 1 00000100:00100000:3.0:1789760782.436238:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495928064:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760782.436319:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27029504, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd009 00010000:00010000:3.0:1789760782.436329:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547c3d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.436345:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760782.436350:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.436357:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547c3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.436372:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547c3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.486616:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973916 is committed 00002000:00010000:2.0:1789760782.486794:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27029504 -> 27033600 00010000:00010000:2.0:1789760782.486809:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547c3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.486822:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547c3d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.486831:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547c3d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.486844:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760782.486849:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547c3d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.486858:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547c3d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.487043:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547c3d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.487053:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547c3d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.487253:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495928064:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51014us (52250us total) trans 4294973916 rc 0/0 00000100:00100000:2.0:1789760782.487264:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6628 00000100:00100000:2.0:1789760782.487274:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766697942685 time 766756941594 state 0x2 ref 0 00000100:00100000:2.0:1789760782.492152:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760782.492256:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495928192 00000100:00100000:3.0:1789760782.492267:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495928192 00000100:00100000:3.0:1789760782.492475:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766697942685 time 766756941594 state 0x0 ref 1 00000100:00100000:3.0:1789760782.492492:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766757202010 time 766756941594 state 0x1 ref 1 00000100:00100000:3.0:1789760782.492531:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766757202010 time 766813247275 state 0x0 ref 1 00000100:00100000:3.0:1789760782.492544:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495928192:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760782.492597:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27033600, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd009 00010000:00010000:3.0:1789760782.492606:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c44 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.492622:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760782.492627:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.492635:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.492649:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760782.519693:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27033600 -> 27037696 00010000:00010000:0.0:1789760782.519723:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.519746:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c44 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.519755:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c44 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.519763:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760782.519768:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c44 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.519777:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c44 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.519797:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c44 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.519806:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547c44 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.519898:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495928192:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27351us (27749us total) trans 4294973917 rc 0/0 00000100:00100000:0.0:1789760782.519908:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6629 00000100:00100000:0.0:1789760782.519918:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766757202010 time 766813247275 state 0x2 ref 0 00000001:00080000:1.0:1789760782.520526:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973917 is committed 00000100:00100000:1.0:1789760782.523168:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760782.523230:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495928320 00000100:00100000:2.0:1789760782.523239:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495928320 00000100:00100000:2.0:1789760782.523311:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766757202010 time 766813247275 state 0x0 ref 1 00000100:00100000:2.0:1789760782.523322:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766813507691 time 766813247275 state 0x1 ref 1 00000100:00100000:2.0:1789760782.523345:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766813507691 time 766844061810 state 0x0 ref 1 00000100:00100000:2.0:1789760782.523360:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495928320:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760782.523404:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27037696, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd009 00010000:00010000:2.0:1789760782.523413:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547c4b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.523429:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760782.523433:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.523439:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547c4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.523453:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547c4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.547307:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973918 is committed 00002000:00010000:3.0:1789760782.547371:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27037696 -> 27041792 00010000:00010000:3.0:1789760782.547386:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547c4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.547399:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547c4b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.547409:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547c4b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.547416:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760782.547421:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547c4b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.547431:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547c4b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.547450:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547c4b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.547458:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547c4b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.547569:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495928320:12345-192.168.203.9@tcp:4:dd.0 Request processed in 24208us (24410us total) trans 4294973918 rc 0/0 00000100:00100000:3.0:1789760782.547579:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6630 00000100:00100000:3.0:1789760782.547588:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766813507691 time 766844061810 state 0x2 ref 0 00000100:00100000:2.0:1789760782.551726:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760782.551813:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495928448 00000100:00100000:3.0:1789760782.551821:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495928448 00000100:00100000:3.0:1789760782.551864:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766813507691 time 766844061810 state 0x0 ref 1 00000100:00100000:3.0:1789760782.551875:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766844322226 time 766844061810 state 0x1 ref 1 00000100:00100000:3.0:1789760782.551900:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766844322226 time 766872616782 state 0x0 ref 1 00000100:00100000:3.0:1789760782.551911:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495928448:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760782.551958:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27041792, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd009 00010000:00010000:3.0:1789760782.551966:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c52 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.551979:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760782.551983:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.551987:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.551996:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.573767:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973919 is committed 00002000:00010000:3.0:1789760782.573984:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27041792 -> 27045888 00010000:00010000:3.0:1789760782.574000:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.574013:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c52 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.574022:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c52 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.574029:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760782.574034:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c52 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.574043:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c52 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.574067:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c52 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.574075:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c52 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.574163:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495928448:12345-192.168.203.9@tcp:4:dd.0 Request processed in 22249us (22450us total) trans 4294973919 rc 0/0 00000100:00100000:3.0:1789760782.574173:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6631 00000100:00100000:3.0:1789760782.574183:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766844322226 time 766872616782 state 0x2 ref 0 00000100:00100000:2.0:1789760782.579978:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760782.580104:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495928576 00000100:00100000:0.0:1789760782.580112:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495928576 00000100:00100000:0.0:1789760782.580165:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766844322226 time 766872616782 state 0x0 ref 1 00000100:00100000:0.0:1789760782.580180:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766872877198 time 766872616782 state 0x1 ref 1 00000100:00100000:0.0:1789760782.580260:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766872877198 time 766900975942 state 0x0 ref 1 00000100:00100000:0.0:1789760782.580273:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495928576:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760782.580348:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27045888, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd009 00010000:00010000:0.0:1789760782.580358:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547c59 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.580379:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:0.0:1789760782.580384:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.580391:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547c59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.580405:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547c59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.627122:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973920 is committed 00002000:00010000:0.0:1789760782.627203:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27045888 -> 27049984 00010000:00010000:0.0:1789760782.627219:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547c59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.627232:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547c59 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.627242:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547c59 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.627249:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ae00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760782.627253:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547c59 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.627262:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547c59 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.627302:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547c59 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.627310:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547c59 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.627441:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495928576:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47166us (47467us total) trans 4294973920 rc 0/0 00000100:00100000:0.0:1789760782.627451:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6632 00000100:00100000:0.0:1789760782.627460:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766872877198 time 766900975942 state 0x2 ref 0 00000100:00100000:1.0:1789760782.634022:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760782.634919:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495928704 00000100:00100000:3.0:1789760782.634931:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495928704 00000100:00100000:3.0:1789760782.635058:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766872877198 time 766900975942 state 0x0 ref 1 00000100:00100000:3.0:1789760782.635074:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766901236358 time 766900975942 state 0x1 ref 1 00000100:00100000:3.0:1789760782.635105:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766901236358 time 766955822112 state 0x0 ref 1 00000100:00100000:3.0:1789760782.635119:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929018a80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495928704:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760782.635254:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27049984, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd009 00010000:00010000:3.0:1789760782.635262:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c60 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.635292:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760782.635295:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.635300:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.635309:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.671412:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973921 is committed 00002000:00010000:0.0:1789760782.671541:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27049984 -> 27054080 00002000:00010000:0.0:1789760782.671546:0:13059:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760781 -> 1789760782 00002000:00010000:0.0:1789760782.671548:0:13059:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760781 -> 1789760782 00010000:00010000:0.0:1789760782.671561:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.671575:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c60 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.671584:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c60 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.671592:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760782.671597:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c60 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.671607:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c60 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.671627:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c60 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.671636:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547c60 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.672326:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929018a80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495928704:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37161us (38264us total) trans 4294973921 rc 0/0 00000100:00100000:0.0:1789760782.672337:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6633 00000100:00100000:0.0:1789760782.672346:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766901236358 time 766955822112 state 0x2 ref 0 00000100:00100000:2.0:1789760782.675988:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760782.676092:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495928832 00000100:00100000:3.0:1789760782.676103:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495928832 00000100:00100000:3.0:1789760782.676155:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766901236358 time 766955822112 state 0x0 ref 1 00000100:00100000:3.0:1789760782.676172:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766956082528 time 766955822112 state 0x1 ref 1 00000100:00100000:3.0:1789760782.676205:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766956082528 time 766996921217 state 0x0 ref 1 00000100:00100000:3.0:1789760782.676217:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495928832:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760782.676269:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27054080, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd009 00010000:00010000:3.0:1789760782.676322:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547c67 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.676339:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760782.676344:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.676351:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547c67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.676371:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547c67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.717535:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973922 is committed 00002000:00010000:2.0:1789760782.718887:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27054080 -> 27058176 00010000:00010000:2.0:1789760782.718908:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547c67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.718921:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547c67 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.718930:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547c67 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.718943:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760782.718948:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547c67 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.718956:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547c67 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.718976:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547c67 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.718983:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547c67 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.721988:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495928832:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45749us (45986us total) trans 4294973922 rc 0/0 00000100:00100000:3.0:1789760782.722003:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6634 00000100:00100000:3.0:1789760782.722014:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766956082528 time 766996921217 state 0x2 ref 0 00000100:00100000:2.0:1789760782.725871:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760782.726782:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495928960 00000100:00100000:0.0:1789760782.726800:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495928960 00000100:00100000:0.0:1789760782.726859:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766956082528 time 766996921217 state 0x0 ref 1 00000100:00100000:0.0:1789760782.726879:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766997181633 time 766996921217 state 0x1 ref 1 00000100:00100000:0.0:1789760782.726910:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766997181633 time 767047627303 state 0x0 ref 1 00000100:00100000:0.0:1789760782.726922:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495928960:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760782.726970:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27058176, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd009 00010000:00010000:0.0:1789760782.726979:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547c6e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.727000:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:0.0:1789760782.727004:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.727011:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547c6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.727024:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547c6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760782.788540:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27058176 -> 27062272 00010000:00010000:1.0:1789760782.788560:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547c6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.788561:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973923 is committed 00010000:00010000:1.0:1789760782.788576:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547c6e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.788587:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547c6e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.788594:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ae00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760782.788600:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547c6e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.788610:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547c6e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.788630:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547c6e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.788639:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547c6e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.788749:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495928960:12345-192.168.203.9@tcp:4:dd.0 Request processed in 61814us (62874us total) trans 4294973923 rc 0/0 00000100:00100000:1.0:1789760782.788760:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6635 00000100:00100000:1.0:1789760782.788778:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766997181633 time 767047627303 state 0x2 ref 0 00000100:00100000:1.0:1789760782.797959:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760782.798034:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495929088 00000100:00100000:0.0:1789760782.798043:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495929088 00000100:00100000:0.0:1789760782.798104:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 766997181633 time 767047627303 state 0x0 ref 1 00000100:00100000:0.0:1789760782.798119:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767047887719 time 767047627303 state 0x1 ref 1 00000100:00100000:0.0:1789760782.798155:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767047887719 time 767118870758 state 0x0 ref 1 00000100:00100000:0.0:1789760782.798170:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495929088:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760782.798219:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27062272, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd009 00010000:00010000:0.0:1789760782.798228:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547c75 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.798244:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:0.0:1789760782.798249:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.798256:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547c75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.798274:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547c75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.867452:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973924 is committed 00002000:00010000:2.0:1789760782.867645:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27062272 -> 27066368 00010000:00010000:2.0:1789760782.867662:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547c75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.867677:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547c75 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.867685:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547c75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.867693:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760782.867699:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547c75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.867718:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547c75 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.867745:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547c75 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.867752:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547c75 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.867844:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495929088:12345-192.168.203.9@tcp:4:dd.0 Request processed in 69674us (69891us total) trans 4294973924 rc 0/0 00000100:00100000:2.0:1789760782.867855:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6636 00000100:00100000:2.0:1789760782.867866:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767047887719 time 767118870758 state 0x2 ref 0 00000100:00100000:2.0:1789760782.877202:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760782.877422:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495929216 00000100:00100000:3.0:1789760782.877433:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495929216 00000100:00100000:3.0:1789760782.877484:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767047887719 time 767118870758 state 0x0 ref 1 00000100:00100000:3.0:1789760782.877498:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767119131174 time 767118870758 state 0x1 ref 1 00000100:00100000:3.0:1789760782.877528:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767119131174 time 767198244651 state 0x0 ref 1 00000100:00100000:3.0:1789760782.877539:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036398850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495929216:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760782.877582:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27066368, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd009 00010000:00010000:3.0:1789760782.877591:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547c7c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.877607:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760782.877611:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.877618:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547c7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.877637:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547c7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.942057:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973925 is committed 00002000:00010000:3.0:1789760782.942140:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27066368 -> 27070464 00010000:00010000:3.0:1789760782.942154:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547c7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.942166:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547c7c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.942174:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547c7c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.942181:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760782.942186:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547c7c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.942194:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547c7c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.942214:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547c7c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.942221:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547c7c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.942509:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036398850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495929216:12345-192.168.203.9@tcp:4:dd.0 Request processed in 64966us (65311us total) trans 4294973925 rc 0/0 00000100:00100000:3.0:1789760782.942521:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6637 00000100:00100000:3.0:1789760782.942531:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767119131174 time 767198244651 state 0x2 ref 0 00000100:00100000:2.0:1789760782.951573:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760782.951679:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495929344 00000100:00100000:1.0:1789760782.951692:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495929344 00000100:00100000:1.0:1789760782.951815:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767119131174 time 767198244651 state 0x0 ref 1 00000100:00100000:1.0:1789760782.951831:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767198505067 time 767198244651 state 0x1 ref 1 00000100:00100000:1.0:1789760782.951932:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767198505067 time 767272647514 state 0x0 ref 1 00000100:00100000:1.0:1789760782.951948:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495929344:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760782.951996:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27070464, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd009 00010000:00010000:1.0:1789760782.952004:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547c83 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.952020:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760782.952024:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.952031:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547c83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.952044:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547c83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.985938:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973926 is committed 00002000:00010000:3.0:1789760782.986138:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27070464 -> 27074560 00010000:00010000:3.0:1789760782.986154:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547c83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.986167:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547c83 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.986177:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547c83 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.986184:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760782.986189:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547c83 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.986197:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547c83 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.986217:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547c83 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.986230:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x4936748354547c83 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.986441:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495929344:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34493us (34871us total) trans 4294973926 rc 0/0 00000100:00100000:3.0:1789760782.986452:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6638 00000100:00100000:3.0:1789760782.986462:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767198505067 time 767272647514 state 0x2 ref 0 00000100:00100000:1.0:1789760782.990186:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760782.990341:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495929472 00000100:00100000:2.0:1789760782.990351:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495929472 00000100:00100000:2.0:1789760782.990404:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767198505067 time 767272647514 state 0x0 ref 1 00000100:00100000:2.0:1789760782.990421:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767272907930 time 767272647514 state 0x1 ref 1 00000100:00100000:2.0:1789760782.990453:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767272907930 time 767311169897 state 0x0 ref 1 00000100:00100000:2.0:1789760782.990468:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495929472:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760782.990515:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27074560, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd009 00010000:00010000:2.0:1789760782.990524:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547c8a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.990540:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:2.0:1789760782.990544:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.990551:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547c8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.990564:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547c8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760783.018897:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27074560 -> 27078656 00010000:00010000:1.0:1789760783.018918:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547c8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.018932:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547c8a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.018942:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547c8a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.018950:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760783.018955:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547c8a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.018965:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547c8a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.018985:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547c8a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.018992:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547c8a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.019081:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495929472:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28610us (28900us total) trans 4294973927 rc 0/0 00000100:00100000:1.0:1789760783.019092:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6639 00000100:00100000:1.0:1789760783.019102:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767272907930 time 767311169897 state 0x2 ref 0 00000001:00080000:3.0:1789760783.019546:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973927 is committed 00000100:00100000:1.0:1789760783.022483:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760783.022626:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495929600 00000100:00100000:3.0:1789760783.022635:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495929600 00000100:00100000:2.0:1789760783.022955:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767272907930 time 767311169897 state 0x0 ref 1 00000100:00100000:2.0:1789760783.022970:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767311430313 time 767311169897 state 0x1 ref 1 00000100:00100000:2.0:1789760783.023000:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767311430313 time 767343716188 state 0x0 ref 1 00000100:00100000:2.0:1789760783.023012:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495929600:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760783.023164:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27078656, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd009 00010000:00010000:2.0:1789760783.023175:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c91 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.023190:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:2.0:1789760783.023195:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.023201:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.023215:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760783.023693:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789760783.058276:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27078656 -> 27082752 00010000:00010000:3.0:1789760783.058325:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.058338:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c91 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.058347:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.058355:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760783.058360:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.058370:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c91 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.058391:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c91 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.058398:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547c91 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.058484:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495929600:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35469us (36004us total) trans 4294973928 rc 0/0 00000100:00100000:3.0:1789760783.058552:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6640 00000100:00100000:3.0:1789760783.058563:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767311430313 time 767343716188 state 0x2 ref 0 00000001:00080000:1.0:1789760783.058633:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973928 is committed 00000100:00100000:2.0:1789760783.062479:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760783.062950:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495929728 00000100:00100000:3.0:1789760783.062960:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495929728 00000100:00100000:3.0:1789760783.063015:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767311430313 time 767343716188 state 0x0 ref 1 00000100:00100000:3.0:1789760783.063030:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767343976604 time 767343716188 state 0x1 ref 1 00000100:00100000:3.0:1789760783.063131:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767343976604 time 767383846873 state 0x0 ref 1 00000100:00100000:3.0:1789760783.063144:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495929728:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760783.063188:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27082752, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd009 00010000:00010000:3.0:1789760783.063196:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547c98 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.063213:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760783.063218:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.063225:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547c98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.063238:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547c98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.101609:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973929 is committed 00002000:00010000:3.0:1789760783.101721:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27082752 -> 27086848 00010000:00010000:3.0:1789760783.101744:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547c98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.101859:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547c98 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.101869:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547c98 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.101876:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760783.101880:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547c98 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.101898:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547c98 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.101916:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547c98 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.101924:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547c98 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.102017:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495929728:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38869us (39542us total) trans 4294973929 rc 0/0 00000100:00100000:3.0:1789760783.102027:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6641 00000100:00100000:3.0:1789760783.102036:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767343976604 time 767383846873 state 0x2 ref 0 00000100:00100000:2.0:1789760783.105620:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760783.105723:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495929856 00000100:00100000:1.0:1789760783.105746:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495929856 00000100:00100000:1.0:1789760783.105959:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767343976604 time 767383846873 state 0x0 ref 1 00000100:00100000:1.0:1789760783.105976:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767384107289 time 767383846873 state 0x1 ref 1 00000100:00100000:1.0:1789760783.106063:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767384107289 time 767426778028 state 0x0 ref 1 00000100:00100000:1.0:1789760783.106076:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495929856:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760783.106124:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27086848, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd009 00010000:00010000:1.0:1789760783.106133:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547c9f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.106149:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760783.106153:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.106160:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547c9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.106174:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547c9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760783.137692:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27086848 -> 27090944 00010000:00010000:0.0:1789760783.137782:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547c9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.137797:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547c9f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.137807:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547c9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.137814:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760783.137820:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547c9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.137830:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547c9f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.137849:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547c9f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.137857:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547c9f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.137985:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495929856:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31905us (32368us total) trans 4294973930 rc 0/0 00000100:00100000:0.0:1789760783.137996:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6642 00000100:00100000:0.0:1789760783.138005:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767384107289 time 767426778028 state 0x2 ref 0 00000001:00080000:0.0:1789760783.138041:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973930 is committed 00000100:00100000:2.0:1789760783.141584:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760783.141770:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495929984 00000100:00100000:3.0:1789760783.141779:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495929984 00000100:00100000:3.0:1789760783.141832:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767384107289 time 767426778028 state 0x0 ref 1 00000100:00100000:3.0:1789760783.141846:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767427038444 time 767426778028 state 0x1 ref 1 00000100:00100000:3.0:1789760783.141878:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767427038444 time 767462595316 state 0x0 ref 1 00000100:00100000:3.0:1789760783.141890:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495929984:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760783.142129:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27090944, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd009 00010000:00010000:3.0:1789760783.142138:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547ca6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.142152:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760783.142157:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.142163:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547ca6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.142177:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547ca6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760783.172967:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27090944 -> 27095040 00010000:00010000:0.0:1789760783.172985:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547ca6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.172999:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547ca6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.173009:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547ca6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.173018:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760783.173023:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547ca6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.173032:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547ca6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.173063:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547ca6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.173077:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547ca6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.173167:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495929984:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31273us (31589us total) trans 4294973931 rc 0/0 00000100:00100000:0.0:1789760783.173177:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6643 00000100:00100000:0.0:1789760783.173188:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767427038444 time 767462595316 state 0x2 ref 0 00000001:00080000:0.0:1789760783.173227:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973931 is committed 00000100:00100000:2.0:1789760783.176588:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760783.176665:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495930112 00000100:00100000:3.0:1789760783.176676:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495930112 00000100:00100000:3.0:1789760783.176803:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767427038444 time 767462595316 state 0x0 ref 1 00000100:00100000:3.0:1789760783.176818:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767462855732 time 767462595316 state 0x1 ref 1 00000100:00100000:3.0:1789760783.176877:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767462855732 time 767497593364 state 0x0 ref 1 00000100:00100000:3.0:1789760783.176890:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495930112:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760783.176936:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27095040, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd009 00010000:00010000:3.0:1789760783.176945:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547cad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.176960:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760783.176964:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.176970:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547cad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.176983:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547cad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.197949:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973932 is committed 00002000:00010000:0.0:1789760783.197996:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27095040 -> 27099136 00010000:00010000:0.0:1789760783.198011:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547cad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.198023:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547cad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.198031:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547cad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.198038:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760783.198043:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547cad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.198053:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547cad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.198085:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547cad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.198091:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547cad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.198162:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495930112:12345-192.168.203.9@tcp:4:dd.0 Request processed in 21270us (21579us total) trans 4294973932 rc 0/0 00000100:00100000:0.0:1789760783.198172:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6644 00000100:00100000:0.0:1789760783.198182:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767462855732 time 767497593364 state 0x2 ref 0 00010000:00010000:2.0:1789760783.198599:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789760783.202272:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760783.202337:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495930240 00000100:00100000:0.0:1789760783.202348:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495930240 00000100:00100000:0.0:1789760783.202398:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767462855732 time 767497593364 state 0x0 ref 1 00000100:00100000:0.0:1789760783.202412:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767497853780 time 767497593364 state 0x1 ref 1 00000100:00100000:0.0:1789760783.202439:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767497853780 time 767523156351 state 0x0 ref 1 00000100:00100000:0.0:1789760783.202451:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495930240:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760783.202506:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27099136, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd009 00010000:00010000:0.0:1789760783.202515:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547cb4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.202530:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:0.0:1789760783.202535:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.202541:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547cb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.202553:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547cb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.230086:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973933 is committed 00002000:00010000:1.0:1789760783.230175:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27099136 -> 27103232 00010000:00010000:1.0:1789760783.230190:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547cb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.230207:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547cb4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.230215:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547cb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.230221:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760783.230225:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547cb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.230234:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547cb4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.230252:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547cb4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.230260:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547cb4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.230473:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495930240:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28019us (28204us total) trans 4294973933 rc 0/0 00000100:00100000:1.0:1789760783.230483:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6645 00000100:00100000:1.0:1789760783.230493:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767497853780 time 767523156351 state 0x2 ref 0 00000100:00100000:2.0:1789760783.243994:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760783.244184:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495930368 00000100:00100000:0.0:1789760783.244194:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495930368 00000100:00100000:0.0:1789760783.244248:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767497853780 time 767523156351 state 0x0 ref 1 00000100:00100000:0.0:1789760783.244263:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767523416767 time 767523156351 state 0x1 ref 1 00000100:00100000:0.0:1789760783.244389:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767523416767 time 767565104311 state 0x0 ref 1 00000100:00100000:0.0:1789760783.244402:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495930368:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760783.244460:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27103232, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd009 00010000:00010000:0.0:1789760783.244468:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547cbb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.244484:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:0.0:1789760783.244488:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.244495:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547cbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.244509:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547cbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.285228:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973934 is committed 00002000:00010000:1.0:1789760783.285345:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27103232 -> 27107328 00010000:00010000:1.0:1789760783.285361:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547cbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.285373:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547cbb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.285382:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547cbb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.285390:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760783.285394:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547cbb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.285402:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547cbb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.285423:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547cbb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.285430:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547cbb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.285534:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495930368:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41130us (41545us total) trans 4294973934 rc 0/0 00000100:00100000:1.0:1789760783.285543:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6646 00000100:00100000:1.0:1789760783.285553:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767523416767 time 767565104311 state 0x2 ref 0 00000100:00100000:2.0:1789760783.290394:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760783.290466:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495930496 00000100:00100000:0.0:1789760783.290475:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495930496 00000100:00100000:0.0:1789760783.290543:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767523416767 time 767565104311 state 0x0 ref 1 00000100:00100000:0.0:1789760783.290557:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767565364727 time 767565104311 state 0x1 ref 1 00000100:00100000:0.0:1789760783.290586:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767565364727 time 767611303185 state 0x0 ref 1 00000100:00100000:0.0:1789760783.290599:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495930496:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760783.290646:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27107328, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd009 00010000:00010000:0.0:1789760783.290655:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547cc2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.290669:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:0.0:1789760783.290673:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.290679:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547cc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.290693:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547cc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.338069:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973935 is committed 00002000:00010000:0.0:1789760783.338152:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27107328 -> 27111424 00010000:00010000:0.0:1789760783.338168:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547cc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.338187:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547cc2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.338197:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547cc2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.338204:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760783.338210:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547cc2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.338219:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547cc2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.338238:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547cc2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.338247:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547cc2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.338448:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495930496:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47847us (48058us total) trans 4294973935 rc 0/0 00000100:00100000:0.0:1789760783.338459:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6647 00000100:00100000:0.0:1789760783.338469:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767565364727 time 767611303185 state 0x2 ref 0 00000100:00100000:2.0:1789760783.347464:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760783.347581:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495930624 00000100:00100000:1.0:1789760783.347592:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495930624 00000100:00100000:1.0:1789760783.347650:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767565364727 time 767611303185 state 0x0 ref 1 00000100:00100000:1.0:1789760783.347666:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767611563601 time 767611303185 state 0x1 ref 1 00000100:00100000:1.0:1789760783.347957:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767611563601 time 767668671983 state 0x0 ref 1 00000100:00100000:1.0:1789760783.347971:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495930624:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760783.348021:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27111424, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd009 00010000:00010000:1.0:1789760783.348031:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547cc9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.348048:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760783.348052:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.348059:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547cc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.348073:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547cc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.382309:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973936 is committed 00002000:00010000:1.0:1789760783.382440:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27111424 -> 27115520 00010000:00010000:1.0:1789760783.382455:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547cc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.382468:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547cc9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.382478:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547cc9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.382487:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760783.382492:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547cc9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.382503:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547cc9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.382522:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547cc9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.382530:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547cc9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.382624:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495930624:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34650us (35164us total) trans 4294973936 rc 0/0 00000100:00100000:1.0:1789760783.382635:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6648 00000100:00100000:1.0:1789760783.382646:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767611563601 time 767668671983 state 0x2 ref 0 00000100:00100000:2.0:1789760783.387885:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760783.388008:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495930752 00000100:00100000:1.0:1789760783.388017:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495930752 00000100:00100000:1.0:1789760783.388080:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767611563601 time 767668671983 state 0x0 ref 1 00000100:00100000:1.0:1789760783.388095:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767668932399 time 767668671983 state 0x1 ref 1 00000100:00100000:1.0:1789760783.388127:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767668932399 time 767708843341 state 0x0 ref 1 00000100:00100000:1.0:1789760783.388140:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495930752:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760783.388187:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27115520, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd009 00010000:00010000:1.0:1789760783.388196:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547cd0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.388212:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760783.388217:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.388223:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547cd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.388237:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547cd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.419416:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973937 is committed 00002000:00010000:0.0:1789760783.419780:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27115520 -> 27119616 00010000:00010000:0.0:1789760783.419858:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547cd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.419871:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547cd0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.419881:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547cd0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.419888:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760783.419893:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547cd0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.419963:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547cd0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.420054:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547cd0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.420063:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547cd0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.420207:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495930752:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32064us (32326us total) trans 4294973937 rc 0/0 00000100:00100000:0.0:1789760783.420218:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6649 00000100:00100000:0.0:1789760783.420227:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767668932399 time 767708843341 state 0x2 ref 0 00000100:00100000:2.0:1789760783.436226:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760783.436321:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495930880 00000100:00100000:0.0:1789760783.436329:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495930880 00000100:00100000:0.0:1789760783.436379:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767668932399 time 767708843341 state 0x0 ref 1 00000100:00100000:0.0:1789760783.436393:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767709103757 time 767708843341 state 0x1 ref 1 00000100:00100000:0.0:1789760783.436425:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767709103757 time 767757142935 state 0x0 ref 1 00000100:00100000:0.0:1789760783.436436:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495930880:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760783.436479:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27119616, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd009 00010000:00010000:0.0:1789760783.436487:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547cd7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.436503:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:0.0:1789760783.436512:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.436518:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547cd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.436533:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547cd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.475527:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973938 is committed 00002000:00010000:0.0:1789760783.475916:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27119616 -> 27123712 00010000:00010000:0.0:1789760783.475933:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547cd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.475946:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547cd7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.475956:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547cd7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.475963:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760783.475967:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547cd7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.475977:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547cd7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.475998:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547cd7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.476007:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547cd7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.476104:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495930880:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39664us (39882us total) trans 4294973938 rc 0/0 00000100:00100000:0.0:1789760783.476114:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6650 00000100:00100000:0.0:1789760783.476124:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767709103757 time 767757142935 state 0x2 ref 0 00000100:00100000:2.0:1789760783.482562:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760783.482636:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495931008 00000100:00100000:1.0:1789760783.482655:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495931008 00000100:00100000:1.0:1789760783.482715:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767709103757 time 767757142935 state 0x0 ref 1 00000100:00100000:1.0:1789760783.482745:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767757403351 time 767757142935 state 0x1 ref 1 00000100:00100000:1.0:1789760783.482772:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767757403351 time 767803489428 state 0x0 ref 1 00000100:00100000:1.0:1789760783.482784:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495931008:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760783.482831:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27123712, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd009 00010000:00010000:1.0:1789760783.482840:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547cde lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.482857:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760783.482863:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.482869:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547cde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.482882:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547cde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760783.518586:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27123712 -> 27127808 00010000:00010000:1.0:1789760783.518611:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547cde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.518625:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547cde lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.518635:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547cde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.518643:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760783.518648:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547cde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.518658:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547cde lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.518679:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547cde lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.518687:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547cde lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.518803:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495931008:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36017us (36244us total) trans 4294973939 rc 0/0 00000100:00100000:1.0:1789760783.518815:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6651 00000100:00100000:1.0:1789760783.518825:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767757403351 time 767803489428 state 0x2 ref 0 00000001:00080000:2.0:1789760783.520500:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973939 is committed 00000100:00100000:2.0:1789760783.524407:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760783.524840:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495931136 00000100:00100000:0.0:1789760783.524852:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495931136 00000100:00100000:0.0:1789760783.524908:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767757403351 time 767803489428 state 0x0 ref 1 00000100:00100000:0.0:1789760783.524922:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767803749844 time 767803489428 state 0x1 ref 1 00000100:00100000:0.0:1789760783.524958:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767803749844 time 767845674767 state 0x0 ref 1 00000100:00100000:0.0:1789760783.524972:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495931136:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760783.525021:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27127808, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd009 00010000:00010000:0.0:1789760783.525030:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547ce5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.525048:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:0.0:1789760783.525053:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.525060:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547ce5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.525075:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547ce5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760783.552936:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27127808 -> 27131904 00010000:00010000:0.0:1789760783.552960:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547ce5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.552973:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547ce5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.552982:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547ce5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.552989:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760783.552996:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547ce5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.553005:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547ce5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.553024:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547ce5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.553033:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547ce5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.553127:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495931136:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28153us (28728us total) trans 4294973940 rc 0/0 00000100:00100000:0.0:1789760783.553138:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6652 00000100:00100000:0.0:1789760783.553148:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767803749844 time 767845674767 state 0x2 ref 0 00000001:00080000:2.0:1789760783.553696:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973940 is committed 00000100:00100000:3.0:1789760783.556231:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760783.556497:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495931264 00000100:00100000:0.0:1789760783.556506:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495931264 00000100:00100000:0.0:1789760783.556565:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767803749844 time 767845674767 state 0x0 ref 1 00000100:00100000:0.0:1789760783.556582:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767845935183 time 767845674767 state 0x1 ref 1 00000100:00100000:0.0:1789760783.556654:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767845935183 time 767877370745 state 0x0 ref 1 00000100:00100000:0.0:1789760783.556667:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495931264:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760783.556776:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27131904, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd009 00010000:00010000:0.0:1789760783.556787:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547cec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.556801:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:0.0:1789760783.556805:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.556812:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547cec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.556825:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547cec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.598582:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973941 is committed 00002000:00010000:0.0:1789760783.598796:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27131904 -> 27136000 00010000:00010000:0.0:1789760783.598815:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547cec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.598828:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547cec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.598837:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547cec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.598845:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760783.598850:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547cec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.598859:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547cec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.600780:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547cec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.600794:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547cec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.600986:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495931264:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44316us (44758us total) trans 4294973941 rc 0/0 00000100:00100000:0.0:1789760783.600998:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6653 00000100:00100000:0.0:1789760783.601008:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767845935183 time 767877370745 state 0x2 ref 0 00000100:00100000:1.0:1789760783.607535:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760783.607646:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495931392 00000100:00100000:0.0:1789760783.607656:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495931392 00000100:00100000:0.0:1789760783.607836:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767845935183 time 767877370745 state 0x0 ref 1 00000100:00100000:0.0:1789760783.607853:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767877631161 time 767877370745 state 0x1 ref 1 00000100:00100000:0.0:1789760783.607922:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767877631161 time 767928637466 state 0x0 ref 1 00000100:00100000:0.0:1789760783.607936:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495931392:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760783.607989:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27136000, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd009 00010000:00010000:0.0:1789760783.607999:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547cf3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.608016:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:0.0:1789760783.608021:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.608028:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547cf3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.608043:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547cf3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760783.640760:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27136000 -> 27140096 00010000:00010000:3.0:1789760783.640779:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547cf3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.640797:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547cf3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.640806:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547cf3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.640813:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760783.640818:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547cf3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.640826:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547cf3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.640846:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547cf3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.640853:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547cf3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.640949:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495931392:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33012us (33420us total) trans 4294973942 rc 0/0 00000100:00100000:3.0:1789760783.640959:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6654 00000100:00100000:3.0:1789760783.640968:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767877631161 time 767928637466 state 0x2 ref 0 00000001:00080000:2.0:1789760783.642232:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973942 is committed 00000100:00100000:3.0:1789760783.646584:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff929036f66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520315520:0@lo:400:kworker.0 00000100:00100000:3.0:1789760783.646654:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760783.646721:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789760783.646735:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff929036f65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520315648:0@lo:400:kworker.0 00000100:00100000:3.0:1789760783.646761:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760783.646796:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520315520 00000100:00100000:3.0:1789760783.646804:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520315520 00000100:00100000:3.0:1789760783.646859:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 522 00000100:00100000:3.0:1789760783.646868:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520315520:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760783.646920:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520315520:12345-0@lo:400:kworker.0 Request processed in 53us (266us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760783.646928:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 522 00000100:00080000:3.0:1789760783.646949:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760783.646982:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff929036f66300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520315520:0@lo:400:kworker.0 00000100:00100000:0.0:1789760783.647084:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff92900c7cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:59c4e264-9b75-482b-8813-3d1dc124f384:3317:1876699520315264:0@lo:400:kworker.0 00000100:00100000:0.0:1789760783.647121:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760783.647152:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff929036f64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3317:1876699520315776:0@lo:400:kworker.0 00000100:00100000:0.0:1789760783.647175:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760783.647196:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1789760783.647202:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff92900c7cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3317:1876699520315392:0@lo:400:kworker.0 00000100:00100000:0.0:1789760783.647223:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760783.647239:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff929036f66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3317:1876699520315904:0@lo:400:kworker.0 00000100:00100000:0.0:1789760783.647259:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760783.647310:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520315648 00000100:00100000:0.0:1789760783.647314:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520315648 00000100:00100000:0.0:1789760783.647349:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520315904 00000100:00100000:0.0:1789760783.647352:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520315904 00000100:00100000:0.0:1789760783.647374:0:6709:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 240 00000100:00100000:0.0:1789760783.647381:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520315648:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760783.647418:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520315648:12345-0@lo:400:kworker.0 Request processed in 38us (655us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760783.647425:0:6709:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 240 00000100:00100000:0.0:1789760783.647437:0:6709:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 241 00000100:00100000:0.0:1789760783.647444:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3317:x1876699520315904:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760783.647474:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3317:x1876699520315904:12345-0@lo:400:kworker.0 Request processed in 31us (214us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760783.647496:0:6709:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 241 00000100:00100000:0.0:1789760783.647512:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520315776 00000100:00100000:0.0:1789760783.647515:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520315776 00000100:00100000:0.0:1789760783.647536:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520315392 00000100:00100000:0.0:1789760783.647539:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520315392 00000100:00100000:0.0:1789760783.647556:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 523 00000100:00100000:0.0:1789760783.647562:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f872680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1876699520315776:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760783.647593:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f872680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1876699520315776:12345-0@lo:400:kworker.0 Request processed in 32us (417us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760783.647599:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 523 00000100:00100000:0.0:1789760783.647609:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 524 00000100:00100000:0.0:1789760783.647614:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1876699520315392:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760783.647639:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290370c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1876699520315392:12345-0@lo:400:kworker.0 Request processed in 25us (415us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760783.647644:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 524 00000100:00080000:0.0:1789760783.647675:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789760783.647679:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789760783.647681:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760783.647701:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff92900c7cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3317:1876699520315392:0@lo:400:kworker.0 00000100:00100000:0.0:1789760783.647710:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff929036f66a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3317:1876699520315904:0@lo:400:kworker.0 00000100:00100000:0.0:1789760783.647719:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff929036f64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3317:1876699520315776:0@lo:400:kworker.0 00000100:00100000:0.0:1789760783.647764:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff929036f65880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520315648:0@lo:400:kworker.0 00000100:00100000:3.0:1789760783.648066:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520315264 00000100:00100000:3.0:1789760783.648070:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520315264 00000100:00100000:3.0:1789760783.648107:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 417 00000100:00100000:3.0:1789760783.648116:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290122cdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:59c4e264-9b75-482b-8813-3d1dc124f384+19:3317:x1876699520315264:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760783.648154:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290122cdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:59c4e264-9b75-482b-8813-3d1dc124f384+19:3317:x1876699520315264:12345-0@lo:400:kworker.0 Request processed in 40us (1033us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760783.648160:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 417 00000100:00100000:1.0:1789760783.648314:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760783.648384:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff92900c7cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:59c4e264-9b75-482b-8813-3d1dc124f384:3317:1876699520315264:0@lo:400:kworker.0 00000100:00100000:3.0:1789760783.649022:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495931520 00000100:00100000:3.0:1789760783.649028:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495931520 00000100:00100000:3.0:1789760783.649070:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767877631161 time 767928637466 state 0x0 ref 1 00000100:00100000:3.0:1789760783.649085:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767928897882 time 767928637466 state 0x1 ref 1 00000100:00100000:3.0:1789760783.649113:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767928897882 time 767969829594 state 0x0 ref 1 00000100:00100000:3.0:1789760783.649124:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036e57100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495931520:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760783.649169:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27140096, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd009 00010000:00010000:3.0:1789760783.649178:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547cfa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.649195:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760783.649201:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.649208:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547cfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.649221:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547cfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.726568:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973943 is committed 00002000:00010000:0.0:1789760783.726748:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27140096 -> 27144192 00002000:00010000:0.0:1789760783.726754:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760782 -> 1789760783 00002000:00010000:0.0:1789760783.726757:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760782 -> 1789760783 00010000:00010000:0.0:1789760783.726771:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547cfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.726784:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547cfa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.726793:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547cfa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.726801:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760783.726805:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547cfa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.726819:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547cfa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.726842:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547cfa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.726850:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547cfa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.729420:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036e57100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495931520:12345-192.168.203.9@tcp:4:dd.0 Request processed in 80286us (81130us total) trans 4294973943 rc 0/0 00000100:00100000:1.0:1789760783.729436:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6655 00000100:00100000:1.0:1789760783.729446:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767928897882 time 767969829594 state 0x2 ref 0 00000100:00100000:0.0:1789760783.735075:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760783.735317:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495931648 00000100:00100000:2.0:1789760783.735327:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495931648 00000100:00100000:2.0:1789760783.735381:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767928897882 time 767969829594 state 0x0 ref 1 00000100:00100000:2.0:1789760783.735397:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767970090010 time 767969829594 state 0x1 ref 1 00000100:00100000:2.0:1789760783.735427:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767970090010 time 768056143915 state 0x0 ref 1 00000100:00100000:2.0:1789760783.735439:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495931648:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760783.735486:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27144192, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd009 00010000:00010000:2.0:1789760783.735495:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547d01 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.735512:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760783.735516:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.735523:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547d01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.735537:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547d01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760783.785991:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27144192 -> 27148288 00010000:00010000:1.0:1789760783.786013:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547d01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.786027:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547d01 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.786041:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547d01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.786048:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760783.786053:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547d01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.786063:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547d01 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.786092:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547d01 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.786104:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547d01 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.786112:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973944 is committed 00000100:00100000:1.0:1789760783.786210:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495931648:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50767us (51139us total) trans 4294973944 rc 0/0 00000100:00100000:1.0:1789760783.786221:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6656 00000100:00100000:1.0:1789760783.786231:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767970090010 time 768056143915 state 0x2 ref 0 00000100:00100000:2.0:1789760783.796398:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760783.796486:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495931776 00000100:00100000:0.0:1789760783.796495:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495931776 00000100:00100000:0.0:1789760783.796548:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 767970090010 time 768056143915 state 0x0 ref 1 00000100:00100000:0.0:1789760783.796563:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768056404331 time 768056143915 state 0x1 ref 1 00000100:00100000:0.0:1789760783.796594:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768056404331 time 768117311435 state 0x0 ref 1 00000100:00100000:0.0:1789760783.796606:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495931776:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760783.796650:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27148288, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd009 00010000:00010000:0.0:1789760783.796659:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547d08 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.796674:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760783.796678:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.796684:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547d08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27148288->27152383] (req 27148288->27152383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.796697:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547d08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27148288->27152383] (req 27148288->27152383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.853878:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973945 is committed 00002000:00010000:0.0:1789760783.853912:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27148288 -> 27152384 00010000:00010000:0.0:1789760783.853929:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547d08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27148288->27152383] (req 27148288->27152383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.853943:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547d08 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27148288->27152383] (req 27148288->27152383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.853957:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547d08 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27148288->27152383] (req 27148288->27152383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.853965:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760783.853970:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547d08 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27148288->27152383] (req 27148288->27152383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.853980:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547d08 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27148288->27152383] (req 27148288->27152383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.854000:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547d08 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27148288->27152383] (req 27148288->27152383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.854008:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547d08 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27148288->27152383] (req 27148288->27152383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.854117:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495931776:12345-192.168.203.9@tcp:4:dd.0 Request processed in 57505us (57722us total) trans 4294973945 rc 0/0 00000100:00100000:0.0:1789760783.854127:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6657 00000100:00100000:0.0:1789760783.854138:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768056404331 time 768117311435 state 0x2 ref 0 00000100:00100000:2.0:1789760783.859969:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760783.860166:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495931904 00000100:00100000:1.0:1789760783.860177:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495931904 00000100:00100000:1.0:1789760783.860231:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768056404331 time 768117311435 state 0x0 ref 1 00000100:00100000:1.0:1789760783.860250:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768117571851 time 768117311435 state 0x1 ref 1 00000100:00100000:1.0:1789760783.860315:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768117571851 time 768181031925 state 0x0 ref 1 00000100:00100000:1.0:1789760783.860329:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495931904:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760783.860376:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27152384, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd009 00010000:00010000:1.0:1789760783.860386:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547d0f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.860401:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:1.0:1789760783.860405:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.860412:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547d0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27152384->27156479] (req 27152384->27156479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.860426:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547d0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27152384->27156479] (req 27152384->27156479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.902092:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973946 is committed 00002000:00010000:3.0:1789760783.902255:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27152384 -> 27156480 00010000:00010000:3.0:1789760783.902273:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547d0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27152384->27156479] (req 27152384->27156479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.902440:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547d0f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27152384->27156479] (req 27152384->27156479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.902450:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547d0f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27152384->27156479] (req 27152384->27156479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.902457:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760783.902462:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547d0f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27152384->27156479] (req 27152384->27156479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.902472:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547d0f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27152384->27156479] (req 27152384->27156479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.902500:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547d0f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27152384->27156479] (req 27152384->27156479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.902509:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354547d0f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27152384->27156479] (req 27152384->27156479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.902859:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495931904:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42526us (42892us total) trans 4294973946 rc 0/0 00000100:00100000:2.0:1789760783.902871:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6658 00000100:00100000:2.0:1789760783.902880:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768117571851 time 768181031925 state 0x2 ref 0 00000100:00100000:1.0:1789760783.908689:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760783.908892:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495932032 00000100:00100000:2.0:1789760783.908901:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495932032 00000100:00100000:2.0:1789760783.908948:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768117571851 time 768181031925 state 0x0 ref 1 00000100:00100000:2.0:1789760783.909106:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768181292341 time 768181031925 state 0x1 ref 1 00000100:00100000:2.0:1789760783.909137:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768181292341 time 768229853349 state 0x0 ref 1 00000100:00100000:2.0:1789760783.909151:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036e55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495932032:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760783.909201:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27156480, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd009 00010000:00010000:2.0:1789760783.909211:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547d16 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.909229:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760783.909234:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.909241:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547d16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27156480->27160575] (req 27156480->27160575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.909254:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547d16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27156480->27160575] (req 27156480->27160575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.939333:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973947 is committed 00002000:00010000:0.0:1789760783.939441:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27156480 -> 27160576 00010000:00010000:0.0:1789760783.939458:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547d16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27156480->27160575] (req 27156480->27160575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.939478:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547d16 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27156480->27160575] (req 27156480->27160575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.939487:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547d16 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27156480->27160575] (req 27156480->27160575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.939495:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760783.939500:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547d16 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27156480->27160575] (req 27156480->27160575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.939509:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547d16 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27156480->27160575] (req 27156480->27160575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.939528:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547d16 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27156480->27160575] (req 27156480->27160575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.939536:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547d16 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27156480->27160575] (req 27156480->27160575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.939632:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036e55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495932032:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30479us (30948us total) trans 4294973947 rc 0/0 00000100:00100000:0.0:1789760783.939641:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6659 00000100:00100000:0.0:1789760783.939651:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768181292341 time 768229853349 state 0x2 ref 0 00000100:00100000:1.0:1789760783.947807:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760783.947952:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495932160 00000100:00100000:0.0:1789760783.948034:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495932160 00000100:00100000:0.0:1789760783.948091:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768181292341 time 768229853349 state 0x0 ref 1 00000100:00100000:0.0:1789760783.948105:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768230113765 time 768229853349 state 0x1 ref 1 00000100:00100000:0.0:1789760783.948139:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768230113765 time 768268855814 state 0x0 ref 1 00000100:00100000:0.0:1789760783.948151:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495932160:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760783.948202:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27160576, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd009 00010000:00010000:0.0:1789760783.948211:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547d1d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.948227:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760783.948232:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.948238:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547d1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27160576->27164671] (req 27160576->27164671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.948252:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547d1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27160576->27164671] (req 27160576->27164671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.985923:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973948 is committed 00002000:00010000:2.0:1789760783.986021:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27160576 -> 27164672 00010000:00010000:2.0:1789760783.986037:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547d1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27160576->27164671] (req 27160576->27164671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.986054:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547d1d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27160576->27164671] (req 27160576->27164671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.986062:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547d1d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27160576->27164671] (req 27160576->27164671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.986070:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760783.986074:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547d1d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27160576->27164671] (req 27160576->27164671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.986083:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547d1d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27160576->27164671] (req 27160576->27164671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.986102:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547d1d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27160576->27164671] (req 27160576->27164671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.986110:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547d1d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27160576->27164671] (req 27160576->27164671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.986233:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495932160:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38079us (38431us total) trans 4294973948 rc 0/0 00000100:00100000:2.0:1789760783.986244:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6660 00000100:00100000:2.0:1789760783.986254:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768230113765 time 768268855814 state 0x2 ref 0 00000100:00100000:2.0:1789760783.993192:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760783.993667:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495932288 00000100:00100000:1.0:1789760783.993677:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495932288 00000100:00100000:2.0:1789760783.993880:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768230113765 time 768268855814 state 0x0 ref 1 00000100:00100000:2.0:1789760783.993896:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768269116230 time 768268855814 state 0x1 ref 1 00000100:00100000:2.0:1789760783.993964:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768269116230 time 768314679776 state 0x0 ref 1 00000100:00100000:2.0:1789760783.993977:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495932288:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760783.994030:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27164672, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd009 00010000:00010000:2.0:1789760783.994041:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547d24 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.994058:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760783.994062:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.994068:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547d24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27164672->27168767] (req 27164672->27168767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.994082:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547d24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27164672->27168767] (req 27164672->27168767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760784.031027:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27164672 -> 27168768 00010000:00010000:0.0:1789760784.031047:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547d24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27164672->27168767] (req 27164672->27168767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.031100:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547d24 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27164672->27168767] (req 27164672->27168767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.031115:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547d24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27164672->27168767] (req 27164672->27168767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.031123:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760784.031128:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547d24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27164672->27168767] (req 27164672->27168767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.031137:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547d24 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27164672->27168767] (req 27164672->27168767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.031156:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547d24 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27164672->27168767] (req 27164672->27168767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.031164:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547d24 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27164672->27168767] (req 27164672->27168767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.031258:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495932288:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37279us (38072us total) trans 4294973949 rc 0/0 00000100:00100000:0.0:1789760784.031268:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6661 00000100:00100000:0.0:1789760784.031309:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768269116230 time 768314679776 state 0x2 ref 0 00000001:00080000:1.0:1789760784.032559:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973949 is committed 00000100:00100000:1.0:1789760784.039240:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760784.039974:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495932416 00000100:00100000:0.0:1789760784.039985:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495932416 00000100:00100000:0.0:1789760784.040044:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768269116230 time 768314679776 state 0x0 ref 1 00000100:00100000:0.0:1789760784.040060:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768314940192 time 768314679776 state 0x1 ref 1 00000100:00100000:0.0:1789760784.040142:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768314940192 time 768360858062 state 0x0 ref 1 00000100:00100000:0.0:1789760784.040155:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495932416:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760784.040205:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27168768, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd009 00010000:00010000:0.0:1789760784.040214:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547d2b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.040230:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760784.040235:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.040241:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547d2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27168768->27172863] (req 27168768->27172863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.040254:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547d2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27168768->27172863] (req 27168768->27172863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.076374:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973950 is committed 00002000:00010000:0.0:1789760784.076574:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27168768 -> 27172864 00010000:00010000:0.0:1789760784.076589:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547d2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27168768->27172863] (req 27168768->27172863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.076681:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547d2b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27168768->27172863] (req 27168768->27172863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.076692:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547d2b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27168768->27172863] (req 27168768->27172863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.076770:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760784.076777:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547d2b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27168768->27172863] (req 27168768->27172863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.076788:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547d2b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27168768->27172863] (req 27168768->27172863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.076808:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547d2b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27168768->27172863] (req 27168768->27172863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.076816:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547d2b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27168768->27172863] (req 27168768->27172863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.076911:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495932416:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36753us (37845us total) trans 4294973950 rc 0/0 00000100:00100000:0.0:1789760784.076921:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6662 00000100:00100000:0.0:1789760784.077263:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768314940192 time 768360858062 state 0x2 ref 0 00000100:00100000:2.0:1789760784.082198:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760784.082308:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495932544 00000100:00100000:3.0:1789760784.082318:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495932544 00000100:00100000:3.0:1789760784.082371:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768314940192 time 768360858062 state 0x0 ref 1 00000100:00100000:3.0:1789760784.082386:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768361118478 time 768360858062 state 0x1 ref 1 00000100:00100000:3.0:1789760784.082405:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768361118478 time 768403122520 state 0x0 ref 1 00000100:00100000:3.0:1789760784.082414:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495932544:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760784.082458:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27172864, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd009 00010000:00010000:3.0:1789760784.082467:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547d32 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.082483:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760784.082488:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.082494:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547d32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27172864->27176959] (req 27172864->27176959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.082509:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547d32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27172864->27176959] (req 27172864->27176959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.118057:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973951 is committed 00002000:00010000:1.0:1789760784.118236:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27172864 -> 27176960 00010000:00010000:1.0:1789760784.118251:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547d32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27172864->27176959] (req 27172864->27176959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.118266:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547d32 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27172864->27176959] (req 27172864->27176959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.118274:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547d32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27172864->27176959] (req 27172864->27176959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.118519:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760784.118524:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547d32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27172864->27176959] (req 27172864->27176959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.118535:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547d32 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27172864->27176959] (req 27172864->27176959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.118686:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547d32 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27172864->27176959] (req 27172864->27176959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.118695:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547d32 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27172864->27176959] (req 27172864->27176959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.118798:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495932544:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36379us (36607us total) trans 4294973951 rc 0/0 00000100:00100000:1.0:1789760784.118809:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6663 00000100:00100000:1.0:1789760784.118819:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768361118478 time 768403122520 state 0x2 ref 0 00000100:00100000:2.0:1789760784.123875:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760784.124122:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495932672 00000100:00100000:0.0:1789760784.124131:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495932672 00000100:00100000:0.0:1789760784.124185:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768361118478 time 768403122520 state 0x0 ref 1 00000100:00100000:0.0:1789760784.124200:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768403382936 time 768403122520 state 0x1 ref 1 00000100:00100000:0.0:1789760784.124232:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768403382936 time 768444949262 state 0x0 ref 1 00000100:00100000:0.0:1789760784.124243:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495932672:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760784.124411:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27176960, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd009 00010000:00010000:0.0:1789760784.124420:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d39 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.124435:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760784.124439:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.124446:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27176960->27181055] (req 27176960->27181055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.124458:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27176960->27181055] (req 27176960->27181055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.156457:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973952 is committed 00002000:00010000:0.0:1789760784.156669:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27176960 -> 27181056 00010000:00010000:0.0:1789760784.156684:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27176960->27181055] (req 27176960->27181055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.156696:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d39 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27176960->27181055] (req 27176960->27181055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.156716:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27176960->27181055] (req 27176960->27181055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.156732:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760784.156737:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27176960->27181055] (req 27176960->27181055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.156747:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d39 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27176960->27181055] (req 27176960->27181055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.156770:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d39 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27176960->27181055] (req 27176960->27181055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.156778:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d39 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27176960->27181055] (req 27176960->27181055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.156900:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495932672:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32653us (33030us total) trans 4294973952 rc 0/0 00000100:00100000:0.0:1789760784.156910:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6664 00000100:00100000:0.0:1789760784.156919:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768403382936 time 768444949262 state 0x2 ref 0 00000100:00100000:2.0:1789760784.159990:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff92900c7cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520316032:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760784.160073:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760784.160270:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520316032 00000100:00100000:0.0:1789760784.160299:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520316032 00000100:00100000:0.0:1789760784.160344:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 216 00000100:00100000:0.0:1789760784.160352:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520316032:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789760784.160386:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760784.160430:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520316032:12345-0@lo:13:osp-pre-1-0.0 Request processed in 78us (357us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760784.160436:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 216 00000100:00100000:2.0:1789760784.160513:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff92900c7cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520316032:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760784.167559:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760784.167660:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495932800 00000100:00100000:2.0:1789760784.167669:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495932800 00000100:00100000:2.0:1789760784.167729:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768403382936 time 768444949262 state 0x0 ref 1 00000100:00100000:2.0:1789760784.167745:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768445209678 time 768444949262 state 0x1 ref 1 00000100:00100000:2.0:1789760784.167775:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768445209678 time 768488492170 state 0x0 ref 1 00000100:00100000:2.0:1789760784.167787:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495932800:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760784.167832:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27181056, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd009 00010000:00010000:2.0:1789760784.167840:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547d40 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.167855:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760784.167860:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.167866:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547d40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27181056->27185151] (req 27181056->27185151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.167880:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547d40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27181056->27185151] (req 27181056->27185151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.213618:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973953 is committed 00002000:00010000:0.0:1789760784.213808:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27181056 -> 27185152 00010000:00010000:0.0:1789760784.213823:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547d40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27181056->27185151] (req 27181056->27185151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.213835:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547d40 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27181056->27185151] (req 27181056->27185151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.213845:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547d40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27181056->27185151] (req 27181056->27185151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.213852:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760784.213858:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547d40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27181056->27185151] (req 27181056->27185151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.213866:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547d40 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27181056->27185151] (req 27181056->27185151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.213887:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547d40 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27181056->27185151] (req 27181056->27185151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.213895:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547d40 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27181056->27185151] (req 27181056->27185151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.213994:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495932800:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46203us (46438us total) trans 4294973953 rc 0/0 00000100:00100000:0.0:1789760784.214005:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6665 00000100:00100000:0.0:1789760784.214015:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768445209678 time 768488492170 state 0x2 ref 0 00000100:00100000:1.0:1789760784.217952:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760784.218068:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495932928 00000100:00100000:3.0:1789760784.218078:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495932928 00000100:00100000:3.0:1789760784.218131:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768445209678 time 768488492170 state 0x0 ref 1 00000100:00100000:3.0:1789760784.218146:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768488752586 time 768488492170 state 0x1 ref 1 00000100:00100000:3.0:1789760784.218175:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768488752586 time 768538891746 state 0x0 ref 1 00000100:00100000:3.0:1789760784.218187:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495932928:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760784.218233:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27185152, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd009 00010000:00010000:3.0:1789760784.218252:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547d47 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.218267:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760784.218272:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.218505:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547d47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27185152->27189247] (req 27185152->27189247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.218520:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547d47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27185152->27189247] (req 27185152->27189247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.226365:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760784.244702:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973954 is committed 00002000:00010000:0.0:1789760784.244829:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27185152 -> 27189248 00010000:00010000:0.0:1789760784.244844:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547d47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27185152->27189247] (req 27185152->27189247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.244863:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547d47 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27185152->27189247] (req 27185152->27189247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.244872:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547d47 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27185152->27189247] (req 27185152->27189247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.244880:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760784.244884:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547d47 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27185152->27189247] (req 27185152->27189247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.244894:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547d47 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27185152->27189247] (req 27185152->27189247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.244914:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547d47 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27185152->27189247] (req 27185152->27189247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.244922:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547d47 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27185152->27189247] (req 27185152->27189247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.245198:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495932928:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27007us (27249us total) trans 4294973954 rc 0/0 00000100:00100000:0.0:1789760784.245210:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6666 00000100:00100000:0.0:1789760784.245220:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768488752586 time 768538891746 state 0x2 ref 0 00000100:00100000:2.0:1789760784.249548:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760784.249633:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495933056 00000100:00100000:0.0:1789760784.249652:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495933056 00000100:00100000:0.0:1789760784.249708:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768488752586 time 768538891746 state 0x0 ref 1 00000100:00100000:0.0:1789760784.249734:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768539152162 time 768538891746 state 0x1 ref 1 00000100:00100000:0.0:1789760784.249767:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768539152162 time 768570483072 state 0x0 ref 1 00000100:00100000:0.0:1789760784.249779:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495933056:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760784.250145:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27189248, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd009 00010000:00010000:0.0:1789760784.250155:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d4e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.250172:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760784.250177:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.250184:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27189248->27193343] (req 27189248->27193343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.250197:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27189248->27193343] (req 27189248->27193343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760784.287229:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27189248 -> 27193344 00010000:00010000:0.0:1789760784.287247:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27189248->27193343] (req 27189248->27193343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.287260:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d4e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27189248->27193343] (req 27189248->27193343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.287274:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27189248->27193343] (req 27189248->27193343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.287313:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760784.287319:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27189248->27193343] (req 27189248->27193343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.287328:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d4e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27189248->27193343] (req 27189248->27193343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.287347:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d4e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27189248->27193343] (req 27189248->27193343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.287354:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d4e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27189248->27193343] (req 27189248->27193343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.287445:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495933056:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37662us (37901us total) trans 4294973955 rc 0/0 00000100:00100000:0.0:1789760784.287455:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6667 00000100:00100000:0.0:1789760784.287465:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768539152162 time 768570483072 state 0x2 ref 0 00000001:00080000:3.0:1789760784.287742:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973955 is committed 00000100:00100000:2.0:1789760784.294875:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760784.294985:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495933184 00000100:00100000:0.0:1789760784.294996:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495933184 00000100:00100000:0.0:1789760784.295050:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768539152162 time 768570483072 state 0x0 ref 1 00000100:00100000:0.0:1789760784.295064:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768570743488 time 768570483072 state 0x1 ref 1 00000100:00100000:0.0:1789760784.295096:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768570743488 time 768615813180 state 0x0 ref 1 00000100:00100000:0.0:1789760784.295108:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495933184:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760784.295161:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27193344, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd009 00010000:00010000:0.0:1789760784.295170:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d55 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.295187:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760784.295192:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.295199:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27193344->27197439] (req 27193344->27197439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.295212:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27193344->27197439] (req 27193344->27197439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760784.324891:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27193344 -> 27197440 00010000:00010000:3.0:1789760784.324911:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27193344->27197439] (req 27193344->27197439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.324924:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d55 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27193344->27197439] (req 27193344->27197439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.324932:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27193344->27197439] (req 27193344->27197439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.324939:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760784.324944:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27193344->27197439] (req 27193344->27197439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.324954:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d55 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27193344->27197439] (req 27193344->27197439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.324973:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d55 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27193344->27197439] (req 27193344->27197439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.324981:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547d55 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27193344->27197439] (req 27193344->27197439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.325073:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495933184:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29962us (30207us total) trans 4294973956 rc 0/0 00000100:00100000:3.0:1789760784.325085:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6668 00000100:00100000:3.0:1789760784.325094:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768570743488 time 768615813180 state 0x2 ref 0 00000001:00080000:2.0:1789760784.325167:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973956 is committed 00000100:00100000:2.0:1789760784.329433:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760784.329524:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495933312 00000100:00100000:0.0:1789760784.329534:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495933312 00000100:00100000:0.0:1789760784.329589:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768570743488 time 768615813180 state 0x0 ref 1 00000100:00100000:0.0:1789760784.329605:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768616073596 time 768615813180 state 0x1 ref 1 00000100:00100000:0.0:1789760784.329637:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768616073596 time 768650353455 state 0x0 ref 1 00000100:00100000:0.0:1789760784.329649:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02768e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495933312:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760784.329698:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27197440, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd009 00010000:00010000:0.0:1789760784.329708:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d5c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.329733:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760784.329738:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.329744:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27197440->27201535] (req 27197440->27201535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.329757:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27197440->27201535] (req 27197440->27201535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.362887:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973957 is committed 00002000:00010000:0.0:1789760784.362996:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27197440 -> 27201536 00010000:00010000:0.0:1789760784.363013:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27197440->27201535] (req 27197440->27201535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.363026:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d5c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27197440->27201535] (req 27197440->27201535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.363034:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d5c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27197440->27201535] (req 27197440->27201535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.363041:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760784.363046:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d5c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27197440->27201535] (req 27197440->27201535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.363056:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d5c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27197440->27201535] (req 27197440->27201535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.363082:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d5c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27197440->27201535] (req 27197440->27201535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.363089:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d5c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27197440->27201535] (req 27197440->27201535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.363207:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02768e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495933312:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33555us (33780us total) trans 4294973957 rc 0/0 00000100:00100000:0.0:1789760784.363217:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6669 00000100:00100000:0.0:1789760784.363227:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768616073596 time 768650353455 state 0x2 ref 0 00000100:00100000:0.0:1789760784.370337:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760784.370440:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495933440 00000100:00100000:2.0:1789760784.370449:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495933440 00000100:00100000:2.0:1789760784.370499:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768616073596 time 768650353455 state 0x0 ref 1 00000100:00100000:2.0:1789760784.370513:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768650613871 time 768650353455 state 0x1 ref 1 00000100:00100000:2.0:1789760784.370532:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768650613871 time 768691249822 state 0x0 ref 1 00000100:00100000:2.0:1789760784.370544:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495933440:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760784.370590:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27201536, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd009 00010000:00010000:2.0:1789760784.370599:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547d63 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.370617:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760784.370621:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.370628:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547d63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27201536->27205631] (req 27201536->27205631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.370642:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547d63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27201536->27205631] (req 27201536->27205631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.404794:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973958 is committed 00002000:00010000:0.0:1789760784.405047:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27201536 -> 27205632 00010000:00010000:0.0:1789760784.405062:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547d63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27201536->27205631] (req 27201536->27205631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.405076:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547d63 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27201536->27205631] (req 27201536->27205631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.405085:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547d63 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27201536->27205631] (req 27201536->27205631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.405093:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760784.405097:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547d63 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27201536->27205631] (req 27201536->27205631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.405123:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547d63 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27201536->27205631] (req 27201536->27205631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.405143:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547d63 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27201536->27205631] (req 27201536->27205631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.405151:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547d63 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27201536->27205631] (req 27201536->27205631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.405240:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495933440:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34693us (34907us total) trans 4294973958 rc 0/0 00000100:00100000:0.0:1789760784.405250:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6670 00000100:00100000:0.0:1789760784.405261:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768650613871 time 768691249822 state 0x2 ref 0 00000100:00100000:2.0:1789760784.415168:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760784.415432:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495933568 00000100:00100000:3.0:1789760784.415445:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495933568 00000100:00100000:3.0:1789760784.415500:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768650613871 time 768691249822 state 0x0 ref 1 00000100:00100000:3.0:1789760784.415516:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768691510238 time 768691249822 state 0x1 ref 1 00000100:00100000:3.0:1789760784.415549:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768691510238 time 768736266009 state 0x0 ref 1 00000100:00100000:3.0:1789760784.415563:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495933568:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760784.415609:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27205632, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd009 00010000:00010000:3.0:1789760784.415618:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547d6a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.415634:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760784.415639:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.415645:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547d6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27205632->27209727] (req 27205632->27209727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.415659:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547d6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27205632->27209727] (req 27205632->27209727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.442481:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973959 is committed 00002000:00010000:0.0:1789760784.442760:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27205632 -> 27209728 00010000:00010000:0.0:1789760784.442832:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547d6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27205632->27209727] (req 27205632->27209727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.442843:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547d6a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27205632->27209727] (req 27205632->27209727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.442851:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547d6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27205632->27209727] (req 27205632->27209727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.442857:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760784.442863:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547d6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27205632->27209727] (req 27205632->27209727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.442871:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547d6a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27205632->27209727] (req 27205632->27209727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.442890:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547d6a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27205632->27209727] (req 27205632->27209727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.442898:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547d6a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27205632->27209727] (req 27205632->27209727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.443023:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495933568:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27459us (27862us total) trans 4294973959 rc 0/0 00000100:00100000:0.0:1789760784.443034:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6671 00000100:00100000:0.0:1789760784.443044:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768691510238 time 768736266009 state 0x2 ref 0 00000100:00100000:2.0:1789760784.454480:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760784.454584:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495933696 00000100:00100000:0.0:1789760784.454594:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495933696 00000100:00100000:0.0:1789760784.454649:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768691510238 time 768736266009 state 0x0 ref 1 00000100:00100000:0.0:1789760784.454665:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768736526425 time 768736266009 state 0x1 ref 1 00000100:00100000:0.0:1789760784.454695:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768736526425 time 768775411820 state 0x0 ref 1 00000100:00100000:0.0:1789760784.454716:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495933696:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760784.454786:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27209728, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd009 00010000:00010000:0.0:1789760784.454797:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d71 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.454813:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760784.454817:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.454823:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27209728->27213823] (req 27209728->27213823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.454838:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27209728->27213823] (req 27209728->27213823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760784.498594:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27209728 -> 27213824 00010000:00010000:0.0:1789760784.498614:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27209728->27213823] (req 27209728->27213823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.498627:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d71 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27209728->27213823] (req 27209728->27213823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.498637:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d71 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27209728->27213823] (req 27209728->27213823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.498643:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760784.498648:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d71 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27209728->27213823] (req 27209728->27213823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.498656:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d71 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27209728->27213823] (req 27209728->27213823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.498679:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d71 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27209728->27213823] (req 27209728->27213823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.498686:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d71 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27209728->27213823] (req 27209728->27213823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.498808:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495933696:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44089us (44332us total) trans 4294973960 rc 0/0 00000100:00100000:0.0:1789760784.498818:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6672 00000100:00100000:0.0:1789760784.498827:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768736526425 time 768775411820 state 0x2 ref 0 00000001:00080000:3.0:1789760784.501732:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973960 is committed 00000100:00100000:3.0:1789760784.502741:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760784.502864:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495933824 00000100:00100000:0.0:1789760784.502872:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495933824 00000100:00100000:0.0:1789760784.502926:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768736526425 time 768775411820 state 0x0 ref 1 00000100:00100000:0.0:1789760784.502940:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768775672236 time 768775411820 state 0x1 ref 1 00000100:00100000:0.0:1789760784.502972:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768775672236 time 768823688904 state 0x0 ref 1 00000100:00100000:0.0:1789760784.502984:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495933824:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760784.503030:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27213824, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd009 00010000:00010000:0.0:1789760784.503039:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547d78 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.503063:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760784.503068:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.503075:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547d78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27213824->27217919] (req 27213824->27217919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.503089:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547d78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27213824->27217919] (req 27213824->27217919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.533837:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973961 is committed 00002000:00010000:0.0:1789760784.533924:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27213824 -> 27217920 00010000:00010000:0.0:1789760784.533940:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547d78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27213824->27217919] (req 27213824->27217919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.533953:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547d78 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27213824->27217919] (req 27213824->27217919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.533963:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547d78 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27213824->27217919] (req 27213824->27217919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.533970:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760784.533975:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547d78 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27213824->27217919] (req 27213824->27217919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.533984:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547d78 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27213824->27217919] (req 27213824->27217919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.534003:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547d78 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27213824->27217919] (req 27213824->27217919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.534016:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547d78 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27213824->27217919] (req 27213824->27217919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.534589:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495933824:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31601us (31847us total) trans 4294973961 rc 0/0 00000100:00100000:1.0:1789760784.534601:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6673 00000100:00100000:1.0:1789760784.534611:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768775672236 time 768823688904 state 0x2 ref 0 00000100:00100000:1.0:1789760784.537609:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760784.537668:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495933952 00000100:00100000:0.0:1789760784.537678:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495933952 00000100:00100000:0.0:1789760784.537741:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768775672236 time 768823688904 state 0x0 ref 1 00000100:00100000:0.0:1789760784.537757:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768823949320 time 768823688904 state 0x1 ref 1 00000100:00100000:0.0:1789760784.537787:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768823949320 time 768858503619 state 0x0 ref 1 00000100:00100000:0.0:1789760784.537800:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495933952:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760784.537845:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27217920, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd009 00010000:00010000:0.0:1789760784.537855:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d7f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.537870:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760784.537874:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.537880:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27217920->27222015] (req 27217920->27222015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.537894:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27217920->27222015] (req 27217920->27222015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760784.568648:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27217920 -> 27222016 00010000:00010000:3.0:1789760784.568667:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27217920->27222015] (req 27217920->27222015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.568681:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d7f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27217920->27222015] (req 27217920->27222015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.568689:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d7f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27217920->27222015] (req 27217920->27222015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.568696:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760784.568709:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d7f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27217920->27222015] (req 27217920->27222015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.568726:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d7f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27217920->27222015] (req 27217920->27222015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.568746:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d7f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27217920->27222015] (req 27217920->27222015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.568754:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547d7f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27217920->27222015] (req 27217920->27222015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.568841:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495933952:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31040us (31237us total) trans 4294973962 rc 0/0 00000100:00100000:3.0:1789760784.568856:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6674 00000100:00100000:3.0:1789760784.568865:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768823949320 time 768858503619 state 0x2 ref 0 00000001:00080000:1.0:1789760784.569873:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973962 is committed 00000100:00100000:2.0:1789760784.576251:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760784.576377:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495934080 00000100:00100000:0.0:1789760784.576386:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495934080 00000100:00100000:0.0:1789760784.576434:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768823949320 time 768858503619 state 0x0 ref 1 00000100:00100000:0.0:1789760784.576448:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768858764035 time 768858503619 state 0x1 ref 1 00000100:00100000:0.0:1789760784.576475:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768858764035 time 768897192151 state 0x0 ref 1 00000100:00100000:0.0:1789760784.576485:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495934080:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760784.576528:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27222016, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd009 00010000:00010000:0.0:1789760784.576537:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547d86 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.576553:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760784.576557:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.576563:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547d86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27222016->27226111] (req 27222016->27226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.576576:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547d86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27222016->27226111] (req 27222016->27226111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.594696:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973963 is committed 00002000:00010000:1.0:1789760784.594774:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27222016 -> 27226112 00010000:00010000:1.0:1789760784.594788:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547d86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27222016->27226111] (req 27222016->27226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.594800:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547d86 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27222016->27226111] (req 27222016->27226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.594809:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547d86 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27222016->27226111] (req 27222016->27226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.594817:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760784.594821:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547d86 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27222016->27226111] (req 27222016->27226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.594830:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547d86 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27222016->27226111] (req 27222016->27226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.594849:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547d86 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27222016->27226111] (req 27222016->27226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.594862:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547d86 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27222016->27226111] (req 27222016->27226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.594959:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495934080:12345-192.168.203.9@tcp:4:dd.0 Request processed in 18469us (18712us total) trans 4294973963 rc 0/0 00000100:00100000:1.0:1789760784.594970:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6675 00000100:00100000:1.0:1789760784.594981:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768858764035 time 768897192151 state 0x2 ref 0 00000100:00100000:0.0:1789760784.598907:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760784.598979:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495934208 00000100:00100000:1.0:1789760784.598990:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495934208 00000100:00100000:1.0:1789760784.599046:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768858764035 time 768897192151 state 0x0 ref 1 00000100:00100000:1.0:1789760784.599061:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768897452567 time 768897192151 state 0x1 ref 1 00000100:00100000:1.0:1789760784.599093:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768897452567 time 768919809588 state 0x0 ref 1 00000100:00100000:1.0:1789760784.599105:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04575500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495934208:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760784.599159:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27226112, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd009 00010000:00010000:1.0:1789760784.599168:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354547d8d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.599183:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:1.0:1789760784.599187:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.599193:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354547d8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27226112->27230207] (req 27226112->27230207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.599211:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354547d8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27226112->27230207] (req 27226112->27230207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.619824:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973964 is committed 00002000:00010000:0.0:1789760784.619902:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27226112 -> 27230208 00010000:00010000:0.0:1789760784.619918:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354547d8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27226112->27230207] (req 27226112->27230207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.619928:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354547d8d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27226112->27230207] (req 27226112->27230207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.619938:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354547d8d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27226112->27230207] (req 27226112->27230207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.619943:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760784.619947:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354547d8d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27226112->27230207] (req 27226112->27230207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.619952:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354547d8d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27226112->27230207] (req 27226112->27230207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.619970:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354547d8d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27226112->27230207] (req 27226112->27230207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.619974:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x4936748354547d8d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27226112->27230207] (req 27226112->27230207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.620063:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04575500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495934208:12345-192.168.203.9@tcp:4:dd.0 Request processed in 20955us (21160us total) trans 4294973964 rc 0/0 00000100:00100000:0.0:1789760784.620073:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6676 00000100:00100000:0.0:1789760784.620083:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768897452567 time 768919809588 state 0x2 ref 0 00000100:00100000:2.0:1789760784.623412:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760784.623501:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495934336 00000100:00100000:3.0:1789760784.623511:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495934336 00000100:00100000:3.0:1789760784.623566:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768897452567 time 768919809588 state 0x0 ref 1 00000100:00100000:3.0:1789760784.623580:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768920070004 time 768919809588 state 0x1 ref 1 00000100:00100000:3.0:1789760784.623609:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768920070004 time 768944326705 state 0x0 ref 1 00000100:00100000:3.0:1789760784.623620:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495934336:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760784.623663:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27230208, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd009 00010000:00010000:3.0:1789760784.623671:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547d94 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.623685:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760784.623689:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.623694:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547d94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27230208->27234303] (req 27230208->27234303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.623706:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547d94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27230208->27234303] (req 27230208->27234303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.652490:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973965 is committed 00002000:00010000:3.0:1789760784.652624:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27230208 -> 27234304 00002000:00010000:3.0:1789760784.652629:0:6714:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760783 -> 1789760784 00002000:00010000:3.0:1789760784.652632:0:6714:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760783 -> 1789760784 00010000:00010000:3.0:1789760784.652645:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547d94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27230208->27234303] (req 27230208->27234303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.652668:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547d94 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27230208->27234303] (req 27230208->27234303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.652678:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547d94 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27230208->27234303] (req 27230208->27234303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.652686:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760784.652691:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547d94 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27230208->27234303] (req 27230208->27234303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.652708:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547d94 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27230208->27234303] (req 27230208->27234303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.652738:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547d94 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27230208->27234303] (req 27230208->27234303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.652746:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547d94 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27230208->27234303] (req 27230208->27234303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.653026:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495934336:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29401us (29616us total) trans 4294973965 rc 0/0 00000100:00100000:3.0:1789760784.653038:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6677 00000100:00100000:3.0:1789760784.653048:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768920070004 time 768944326705 state 0x2 ref 0 00000100:00100000:1.0:1789760784.656390:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760784.656573:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495934464 00000100:00100000:0.0:1789760784.656584:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495934464 00000100:00100000:0.0:1789760784.656640:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768920070004 time 768944326705 state 0x0 ref 1 00000100:00100000:0.0:1789760784.656812:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768944587121 time 768944326705 state 0x1 ref 1 00000100:00100000:0.0:1789760784.656879:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768944587121 time 768977595558 state 0x0 ref 1 00000100:00100000:0.0:1789760784.656892:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495934464:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760784.656940:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27234304, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd009 00010000:00010000:0.0:1789760784.656948:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547d9b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.656963:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760784.656967:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.656973:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547d9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27234304->27238399] (req 27234304->27238399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.656985:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547d9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27234304->27238399] (req 27234304->27238399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.689605:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973966 is committed 00002000:00010000:0.0:1789760784.689804:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27234304 -> 27238400 00010000:00010000:0.0:1789760784.689820:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547d9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27234304->27238399] (req 27234304->27238399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.689834:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547d9b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27234304->27238399] (req 27234304->27238399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.689842:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547d9b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27234304->27238399] (req 27234304->27238399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.689849:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760784.689854:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547d9b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27234304->27238399] (req 27234304->27238399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.689862:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547d9b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27234304->27238399] (req 27234304->27238399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.689881:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547d9b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27234304->27238399] (req 27234304->27238399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.689889:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547d9b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27234304->27238399] (req 27234304->27238399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.690101:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495934464:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33209us (33717us total) trans 4294973966 rc 0/0 00000100:00100000:0.0:1789760784.690109:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6678 00000100:00100000:0.0:1789760784.690117:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768944587121 time 768977595558 state 0x2 ref 0 00000100:00100000:1.0:1789760784.695574:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760784.695729:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495934592 00000100:00100000:2.0:1789760784.695739:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495934592 00000100:00100000:2.0:1789760784.695795:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768944587121 time 768977595558 state 0x0 ref 1 00000100:00100000:2.0:1789760784.695813:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768977855974 time 768977595558 state 0x1 ref 1 00000100:00100000:2.0:1789760784.695930:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768977855974 time 769016645080 state 0x0 ref 1 00000100:00100000:2.0:1789760784.695943:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495934592:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760784.695993:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27238400, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd009 00010000:00010000:2.0:1789760784.696003:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547da2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.696020:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:2.0:1789760784.696025:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.696031:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547da2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27238400->27242495] (req 27238400->27242495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.696045:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547da2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27238400->27242495] (req 27238400->27242495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.719261:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973967 is committed 00002000:00010000:0.0:1789760784.719457:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27238400 -> 27242496 00010000:00010000:0.0:1789760784.719473:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547da2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27238400->27242495] (req 27238400->27242495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.719487:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547da2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27238400->27242495] (req 27238400->27242495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.719496:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547da2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27238400->27242495] (req 27238400->27242495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.719507:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760784.719512:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547da2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27238400->27242495] (req 27238400->27242495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.719520:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547da2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27238400->27242495] (req 27238400->27242495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.719540:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547da2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27238400->27242495] (req 27238400->27242495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.719548:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547da2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27238400->27242495] (req 27238400->27242495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.719655:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495934592:12345-192.168.203.9@tcp:4:dd.0 Request processed in 23706us (24083us total) trans 4294973967 rc 0/0 00000100:00100000:0.0:1789760784.719664:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6679 00000100:00100000:0.0:1789760784.719675:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768977855974 time 769016645080 state 0x2 ref 0 00000100:00100000:2.0:1789760784.727962:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760784.728155:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495934720 00000100:00100000:3.0:1789760784.728168:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495934720 00000100:00100000:3.0:1789760784.728239:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 768977855974 time 769016645080 state 0x0 ref 1 00000100:00100000:3.0:1789760784.728258:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769016905496 time 769016645080 state 0x1 ref 1 00000100:00100000:3.0:1789760784.728342:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769016905496 time 769049058381 state 0x0 ref 1 00000100:00100000:3.0:1789760784.728356:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495934720:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760784.728417:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27242496, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd009 00010000:00010000:3.0:1789760784.728429:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547da9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.728450:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:3.0:1789760784.728455:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.728463:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547da9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27242496->27246591] (req 27242496->27246591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.728478:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547da9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27242496->27246591] (req 27242496->27246591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.776347:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973968 is committed 00002000:00010000:0.0:1789760784.776626:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27242496 -> 27246592 00010000:00010000:0.0:1789760784.776646:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547da9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27242496->27246591] (req 27242496->27246591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.776673:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547da9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27242496->27246591] (req 27242496->27246591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.776690:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547da9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27242496->27246591] (req 27242496->27246591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.777009:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760784.777017:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547da9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27242496->27246591] (req 27242496->27246591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.777029:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547da9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27242496->27246591] (req 27242496->27246591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.777057:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547da9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27242496->27246591] (req 27242496->27246591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.777067:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547da9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27242496->27246591] (req 27242496->27246591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.777200:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495934720:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48839us (49244us total) trans 4294973968 rc 0/0 00000100:00100000:0.0:1789760784.777211:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6680 00000100:00100000:0.0:1789760784.777223:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769016905496 time 769049058381 state 0x2 ref 0 00000100:00100000:1.0:1789760784.780203:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760784.780385:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495934848 00000100:00100000:0.0:1789760784.780400:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495934848 00000100:00100000:0.0:1789760784.780475:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769016905496 time 769049058381 state 0x0 ref 1 00000100:00100000:0.0:1789760784.780496:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769049318797 time 769049058381 state 0x1 ref 1 00000100:00100000:0.0:1789760784.780567:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769049318797 time 769101282891 state 0x0 ref 1 00000100:00100000:0.0:1789760784.780583:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495934848:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760784.780646:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27246592, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd009 00010000:00010000:0.0:1789760784.780658:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547db0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.780690:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760784.780695:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.780704:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547db0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27246592->27250687] (req 27246592->27250687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.780737:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547db0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27246592->27250687] (req 27246592->27250687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.827634:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973969 is committed 00002000:00010000:1.0:1789760784.827891:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27246592 -> 27250688 00010000:00010000:1.0:1789760784.827910:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547db0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27246592->27250687] (req 27246592->27250687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.827926:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547db0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27246592->27250687] (req 27246592->27250687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.827937:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547db0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27246592->27250687] (req 27246592->27250687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.827944:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760784.827951:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547db0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27246592->27250687] (req 27246592->27250687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.827962:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547db0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27246592->27250687] (req 27246592->27250687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.827987:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547db0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27246592->27250687] (req 27246592->27250687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.827995:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547db0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27246592->27250687] (req 27246592->27250687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.828138:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495934848:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47551us (47944us total) trans 4294973969 rc 0/0 00000100:00100000:1.0:1789760784.828149:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6681 00000100:00100000:1.0:1789760784.828161:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769049318797 time 769101282891 state 0x2 ref 0 00000100:00100000:0.0:1789760784.840019:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760784.840154:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495934976 00000100:00100000:2.0:1789760784.840167:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495934976 00000100:00100000:2.0:1789760784.840247:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769049318797 time 769101282891 state 0x0 ref 1 00000100:00100000:2.0:1789760784.840267:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769101543307 time 769101282891 state 0x1 ref 1 00000100:00100000:2.0:1789760784.840411:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769101543307 time 769161126683 state 0x0 ref 1 00000100:00100000:2.0:1789760784.840428:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04577b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495934976:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760784.840498:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27250688, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd009 00010000:00010000:2.0:1789760784.840511:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547db7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.840535:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:2.0:1789760784.840542:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.840551:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547db7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27250688->27254783] (req 27250688->27254783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.840570:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547db7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27250688->27254783] (req 27250688->27254783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.887010:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760784.887069:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760784.887098:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760784.887130:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495935104 00000100:00100000:1.0:1789760784.887143:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495935104 00000100:00100000:1.0:1789760784.887215:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 418 00000100:00100000:1.0:1789760784.887226:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290122cc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2357:x1876699495935104:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:1.0:1789760784.887338:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290122cc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2357:x1876699495935104:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 114us (268us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760784.887346:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 418 00000100:00100000:1.0:1789760784.887385:0:8003:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495935232 00000100:00100000:1.0:1789760784.887389:0:8003:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495935232 00000100:00100000:1.0:1789760784.887430:0:8003:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495935360 00000100:00100000:1.0:1789760784.887433:0:8003:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495935360 00000100:00100000:1.0:1789760784.887458:0:8003:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 525 00000100:00100000:1.0:1789760784.887466:0:8003:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:17022965-1e4f-46fb-915c-61730096b8e2+8:2357:x1876699495935232:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:1.0:1789760784.887500:0:8003:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:17022965-1e4f-46fb-915c-61730096b8e2+8:2357:x1876699495935232:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 35us (507us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760784.887508:0:8003:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 525 00000100:00100000:1.0:1789760784.887519:0:8003:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 526 00000100:00100000:1.0:1789760784.887539:0:8003:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2357:x1876699495935360:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:1.0:1789760784.887564:0:8003:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2357:x1876699495935360:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 39us (466us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760784.887570:0:8003:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 526 00000100:00080000:1.0:1789760784.887592:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:2.0:1789760784.904613:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973970 is committed 00002000:00010000:1.0:1789760784.904978:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27250688 -> 27254784 00010000:00010000:1.0:1789760784.904999:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547db7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27250688->27254783] (req 27250688->27254783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.905015:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547db7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27250688->27254783] (req 27250688->27254783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.905026:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547db7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27250688->27254783] (req 27250688->27254783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.905035:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760784.905040:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547db7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27250688->27254783] (req 27250688->27254783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.905051:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547db7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27250688->27254783] (req 27250688->27254783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.905083:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547db7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27250688->27254783] (req 27250688->27254783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.905093:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547db7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27250688->27254783] (req 27250688->27254783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.905261:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04577b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495934976:12345-192.168.203.9@tcp:4:dd.0 Request processed in 64829us (65250us total) trans 4294973970 rc 0/0 00000100:00100000:1.0:1789760784.905274:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6682 00000100:00100000:1.0:1789760784.905330:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769101543307 time 769161126683 state 0x2 ref 0 00000100:00100000:1.0:1789760784.912514:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760784.916369:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495935488 00000100:00100000:0.0:1789760784.916391:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495935488 00000100:00100000:0.0:1789760784.916479:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769101543307 time 769161126683 state 0x0 ref 1 00000100:00100000:0.0:1789760784.916499:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769161387099 time 769161126683 state 0x1 ref 1 00000100:00100000:0.0:1789760784.916526:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769161387099 time 769237242382 state 0x0 ref 1 00000100:00100000:0.0:1789760784.916541:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495935488:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760784.916612:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27254784, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd009 00010000:00010000:0.0:1789760784.916623:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547dbe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.916642:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760784.916648:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.916657:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547dbe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27254784->27258879] (req 27254784->27258879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.916929:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547dbe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27254784->27258879] (req 27254784->27258879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760784.990429:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27254784 -> 27258880 00010000:00010000:1.0:1789760784.990457:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547dbe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27254784->27258879] (req 27254784->27258879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.990472:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547dbe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27254784->27258879] (req 27254784->27258879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.990482:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547dbe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27254784->27258879] (req 27254784->27258879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.990490:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760784.990496:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547dbe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27254784->27258879] (req 27254784->27258879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.990513:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547dbe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27254784->27258879] (req 27254784->27258879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.990538:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547dbe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27254784->27258879] (req 27254784->27258879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.990548:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547dbe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27254784->27258879] (req 27254784->27258879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.990731:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495935488:12345-192.168.203.9@tcp:4:dd.0 Request processed in 74177us (78434us total) trans 4294973971 rc 0/0 00000100:00100000:1.0:1789760784.990744:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6683 00000001:00080000:0.0:1789760784.990757:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973971 is committed 00000100:00100000:1.0:1789760784.990822:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769161387099 time 769237242382 state 0x2 ref 0 00000100:00100000:0.0:1789760784.994999:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760784.995403:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495935616 00000100:00100000:1.0:1789760784.995417:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495935616 00000100:00100000:1.0:1789760784.995489:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769161387099 time 769237242382 state 0x0 ref 1 00000100:00100000:1.0:1789760784.995509:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769237502798 time 769237242382 state 0x1 ref 1 00000100:00100000:1.0:1789760784.995545:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769237502798 time 769316261018 state 0x0 ref 1 00000100:00100000:1.0:1789760784.995558:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495935616:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760784.995623:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27258880, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd009 00010000:00010000:1.0:1789760784.995635:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547dc5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.995655:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760784.995661:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.995669:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547dc5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27258880->27262975] (req 27258880->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.995686:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547dc5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27258880->27262975] (req 27258880->27262975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.078473:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973972 is committed 00002000:00010000:0.0:1789760785.078593:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27258880 -> 27262976 00010000:00010000:0.0:1789760785.078614:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547dc5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27258880->27262975] (req 27258880->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.078628:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547dc5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27258880->27262975] (req 27258880->27262975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.078638:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547dc5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27258880->27262975] (req 27258880->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.078647:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760785.078653:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547dc5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27258880->27262975] (req 27258880->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.078669:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547dc5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27258880->27262975] (req 27258880->27262975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.078707:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547dc5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27258880->27262975] (req 27258880->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.078726:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354547dc5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27258880->27262975] (req 27258880->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.078973:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495935616:12345-192.168.203.9@tcp:4:dd.0 Request processed in 83408us (83979us total) trans 4294973972 rc 0/0 00000100:00100000:0.0:1789760785.078987:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6684 00000100:00100000:0.0:1789760785.079004:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769237502798 time 769316261018 state 0x2 ref 0 00000100:00100000:2.0:1789760785.085846:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760785.085967:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495935744 00000100:00100000:1.0:1789760785.085981:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495935744 00000100:00100000:1.0:1789760785.086188:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769237502798 time 769316261018 state 0x0 ref 1 00000100:00100000:1.0:1789760785.086209:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769316521434 time 769316261018 state 0x1 ref 1 00000100:00100000:1.0:1789760785.086244:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769316521434 time 769406961057 state 0x0 ref 1 00000100:00100000:1.0:1789760785.086260:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495935744:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760785.086349:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27262976, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd009 00010000:00010000:1.0:1789760785.086360:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547dcc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.086381:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760785.086386:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.086394:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547dcc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27262976->27267071] (req 27262976->27267071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.086417:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547dcc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27262976->27267071] (req 27262976->27267071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760785.087956:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760785.128826:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973973 is committed 00002000:00010000:0.0:1789760785.128969:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27262976 -> 27267072 00002000:00010000:0.0:1789760785.128975:0:13055:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 53257 -> 55305 00010000:00010000:0.0:1789760785.128992:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547dcc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27262976->27267071] (req 27262976->27267071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.129007:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547dcc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27262976->27267071] (req 27262976->27267071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.129016:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547dcc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27262976->27267071] (req 27262976->27267071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.129029:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760785.129035:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547dcc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27262976->27267071] (req 27262976->27267071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.129053:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547dcc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27262976->27267071] (req 27262976->27267071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.129079:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547dcc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27262976->27267071] (req 27262976->27267071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.129086:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547dcc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27262976->27267071] (req 27262976->27267071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.129222:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495935744:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42959us (43527us total) trans 4294973973 rc 0/0 00000100:00100000:0.0:1789760785.129237:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6685 00000100:00100000:0.0:1789760785.129251:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769316521434 time 769406961057 state 0x2 ref 0 00000100:00100000:0.0:1789760785.132722:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760785.132845:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495935872 00000100:00100000:3.0:1789760785.132858:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495935872 00000100:00100000:3.0:1789760785.132926:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769316521434 time 769406961057 state 0x0 ref 1 00000100:00100000:3.0:1789760785.132944:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769407221473 time 769406961057 state 0x1 ref 1 00000100:00100000:3.0:1789760785.132975:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769407221473 time 769453692300 state 0x0 ref 1 00000100:00100000:3.0:1789760785.132988:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495935872:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760785.133057:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27267072, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:3.0:1789760785.133068:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547dd3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.133088:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:3.0:1789760785.133093:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.133101:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547dd3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27267072->27271167] (req 27267072->27271167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.133118:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547dd3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27267072->27271167] (req 27267072->27271167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.188304:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973974 is committed 00002000:00010000:1.0:1789760785.190200:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27267072 -> 27271168 00010000:00010000:1.0:1789760785.190313:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547dd3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27267072->27271167] (req 27267072->27271167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.190329:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547dd3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27267072->27271167] (req 27267072->27271167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.190339:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547dd3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27267072->27271167] (req 27267072->27271167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.190347:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760785.190353:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547dd3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27267072->27271167] (req 27267072->27271167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.190362:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547dd3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27267072->27271167] (req 27267072->27271167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.190385:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547dd3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27267072->27271167] (req 27267072->27271167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.190392:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547dd3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27267072->27271167] (req 27267072->27271167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.190511:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495935872:12345-192.168.203.9@tcp:4:dd.0 Request processed in 57518us (57812us total) trans 4294973974 rc 0/0 00000100:00100000:1.0:1789760785.190523:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6686 00000100:00100000:1.0:1789760785.190536:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769407221473 time 769453692300 state 0x2 ref 0 00000100:00100000:3.0:1789760785.195975:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760785.196189:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495936000 00000100:00100000:0.0:1789760785.196203:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495936000 00000100:00100000:0.0:1789760785.196310:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769407221473 time 769453692300 state 0x0 ref 1 00000100:00100000:0.0:1789760785.196331:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769453952716 time 769453692300 state 0x1 ref 1 00000100:00100000:0.0:1789760785.196363:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769453952716 time 769517079263 state 0x0 ref 1 00000100:00100000:0.0:1789760785.196379:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495936000:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760785.196444:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27271168, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:0.0:1789760785.196455:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547dda lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.196475:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760785.196480:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.196493:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547dda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27271168->27275263] (req 27271168->27275263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.196511:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547dda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27271168->27275263] (req 27271168->27275263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.250663:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760785.255038:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27271168 -> 27275264 00010000:00010000:1.0:1789760785.255067:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547dda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27271168->27275263] (req 27271168->27275263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.255084:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547dda lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27271168->27275263] (req 27271168->27275263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.255095:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547dda lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27271168->27275263] (req 27271168->27275263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.255104:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760785.255110:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547dda lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27271168->27275263] (req 27271168->27275263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.255120:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547dda lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27271168->27275263] (req 27271168->27275263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.255144:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547dda lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27271168->27275263] (req 27271168->27275263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.255158:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547dda lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27271168->27275263] (req 27271168->27275263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.255325:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495936000:12345-192.168.203.9@tcp:4:dd.0 Request processed in 58902us (59319us total) trans 4294973975 rc 0/0 00000100:00100000:1.0:1789760785.255339:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6687 00000100:00100000:1.0:1789760785.255351:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769453952716 time 769517079263 state 0x2 ref 0 00000001:00080000:2.0:1789760785.255914:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973975 is committed 00000100:00100000:2.0:1789760785.258222:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760785.258377:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495936128 00000100:00100000:0.0:1789760785.258391:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495936128 00000100:00100000:0.0:1789760785.258485:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769453952716 time 769517079263 state 0x0 ref 1 00000100:00100000:0.0:1789760785.258507:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769517339679 time 769517079263 state 0x1 ref 1 00000100:00100000:0.0:1789760785.258543:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769517339679 time 769579258819 state 0x0 ref 1 00000100:00100000:0.0:1789760785.258557:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495936128:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760785.258627:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27275264, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:0.0:1789760785.258638:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547de1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.258664:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760785.258669:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.258677:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547de1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27275264->27279359] (req 27275264->27279359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.258707:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547de1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27275264->27279359] (req 27275264->27279359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.329008:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973976 is committed 00002000:00010000:0.0:1789760785.329369:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27275264 -> 27279360 00010000:00010000:0.0:1789760785.329390:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547de1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27275264->27279359] (req 27275264->27279359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.329407:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547de1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27275264->27279359] (req 27275264->27279359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.329417:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547de1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27275264->27279359] (req 27275264->27279359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.329426:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760785.329431:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547de1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27275264->27279359] (req 27275264->27279359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.329442:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547de1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27275264->27279359] (req 27275264->27279359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.329468:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547de1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27275264->27279359] (req 27275264->27279359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.329477:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547de1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27275264->27279359] (req 27275264->27279359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.329823:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495936128:12345-192.168.203.9@tcp:4:dd.0 Request processed in 71258us (71607us total) trans 4294973976 rc 0/0 00000100:00100000:0.0:1789760785.329837:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6688 00000100:00100000:0.0:1789760785.329851:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769517339679 time 769579258819 state 0x2 ref 0 00000100:00100000:0.0:1789760785.337430:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760785.338216:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495936256 00000100:00100000:1.0:1789760785.338232:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495936256 00000100:00100000:1.0:1789760785.338834:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769517339679 time 769579258819 state 0x0 ref 1 00000100:00100000:1.0:1789760785.338856:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769579519235 time 769579258819 state 0x1 ref 1 00000100:00100000:1.0:1789760785.338900:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769579519235 time 769659616418 state 0x0 ref 1 00000100:00100000:1.0:1789760785.338919:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04929500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495936256:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760785.338987:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27279360, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:1.0:1789760785.338999:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547de8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.339021:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760785.339027:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.339035:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547de8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27279360->27283455] (req 27279360->27283455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.339058:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547de8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27279360->27283455] (req 27279360->27283455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760785.383878:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27279360 -> 27283456 00010000:00010000:1.0:1789760785.383907:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547de8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27279360->27283455] (req 27279360->27283455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.383924:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547de8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27279360->27283455] (req 27279360->27283455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.383935:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547de8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27279360->27283455] (req 27279360->27283455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.383944:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760785.383950:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547de8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27279360->27283455] (req 27279360->27283455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.383961:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547de8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27279360->27283455] (req 27279360->27283455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.383986:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547de8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27279360->27283455] (req 27279360->27283455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.383995:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547de8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27279360->27283455] (req 27279360->27283455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.384129:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04929500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495936256:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45209us (46706us total) trans 4294973977 rc 0/0 00000100:00100000:1.0:1789760785.384142:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6689 00000100:00100000:1.0:1789760785.384154:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769579519235 time 769659616418 state 0x2 ref 0 00000001:00080000:3.0:1789760785.385027:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973977 is committed 00000100:00100000:0.0:1789760785.389678:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760785.389878:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495936384 00000100:00100000:1.0:1789760785.389892:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495936384 00000100:00100000:1.0:1789760785.389965:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769579519235 time 769659616418 state 0x0 ref 1 00000100:00100000:1.0:1789760785.389982:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769659876834 time 769659616418 state 0x1 ref 1 00000100:00100000:1.0:1789760785.390077:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769659876834 time 769710791796 state 0x0 ref 1 00000100:00100000:1.0:1789760785.390092:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01c8d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495936384:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760785.390156:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27283456, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:1.0:1789760785.390167:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547def lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.390188:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760785.390194:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.390202:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547def lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27283456->27287551] (req 27283456->27287551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.390217:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547def lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27283456->27287551] (req 27283456->27287551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.424686:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973978 is committed 00002000:00010000:0.0:1789760785.424878:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27283456 -> 27287552 00010000:00010000:0.0:1789760785.424910:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547def lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27283456->27287551] (req 27283456->27287551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.424926:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547def lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27283456->27287551] (req 27283456->27287551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.424936:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547def lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27283456->27287551] (req 27283456->27287551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.424944:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760785.424950:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547def lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27283456->27287551] (req 27283456->27287551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.424959:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547def lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27283456->27287551] (req 27283456->27287551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.424986:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547def lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27283456->27287551] (req 27283456->27287551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.425004:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354547def lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27283456->27287551] (req 27283456->27287551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.425129:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01c8d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495936384:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35034us (35458us total) trans 4294973978 rc 0/0 00000100:00100000:0.0:1789760785.425141:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6690 00000100:00100000:0.0:1789760785.425153:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769659876834 time 769710791796 state 0x2 ref 0 00000100:00100000:2.0:1789760785.432093:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760785.432410:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495936512 00000100:00100000:0.0:1789760785.432425:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495936512 00000100:00100000:0.0:1789760785.432498:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769659876834 time 769710791796 state 0x0 ref 1 00000100:00100000:0.0:1789760785.432513:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769711052212 time 769710791796 state 0x1 ref 1 00000100:00100000:3.0:1789760785.432590:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769711052212 time 769753302845 state 0x0 ref 1 00000100:00100000:3.0:1789760785.432609:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495936512:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760785.432682:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27287552, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:3.0:1789760785.432694:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547df6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.432890:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:3.0:1789760785.432896:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.432905:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547df6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27287552->27291647] (req 27287552->27291647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.432923:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547df6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27287552->27291647] (req 27287552->27291647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760785.484702:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27287552 -> 27291648 00010000:00010000:1.0:1789760785.484739:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547df6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27287552->27291647] (req 27287552->27291647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.484863:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547df6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27287552->27291647] (req 27287552->27291647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.484870:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973979 is committed 00010000:00010000:1.0:1789760785.484874:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547df6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27287552->27291647] (req 27287552->27291647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.484881:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760785.484887:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547df6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27287552->27291647] (req 27287552->27291647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.484897:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547df6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27287552->27291647] (req 27287552->27291647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.484923:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547df6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27287552->27291647] (req 27287552->27291647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.484931:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547df6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27287552->27291647] (req 27287552->27291647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.485169:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495936512:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52557us (53083us total) trans 4294973979 rc 0/0 00000100:00100000:1.0:1789760785.485186:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6691 00000100:00100000:1.0:1789760785.485199:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769711052212 time 769753302845 state 0x2 ref 0 00000100:00100000:1.0:1789760785.499138:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760785.499518:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495936640 00000100:00100000:3.0:1789760785.499531:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495936640 00000100:00100000:3.0:1789760785.499598:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769711052212 time 769753302845 state 0x0 ref 1 00000100:00100000:3.0:1789760785.499617:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769753563261 time 769753302845 state 0x1 ref 1 00000100:00100000:3.0:1789760785.499649:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769753563261 time 769820366412 state 0x0 ref 1 00000100:00100000:3.0:1789760785.499662:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495936640:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760785.499792:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27291648, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:3.0:1789760785.499803:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547dfd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.499823:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:3.0:1789760785.499828:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.499834:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547dfd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27291648->27295743] (req 27291648->27295743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.499857:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547dfd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27291648->27295743] (req 27291648->27295743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.571757:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973980 is committed 00002000:00010000:3.0:1789760785.571970:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27291648 -> 27295744 00010000:00010000:3.0:1789760785.572100:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547dfd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27291648->27295743] (req 27291648->27295743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.572116:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547dfd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27291648->27295743] (req 27291648->27295743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.572126:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547dfd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27291648->27295743] (req 27291648->27295743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.572134:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760785.572140:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547dfd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27291648->27295743] (req 27291648->27295743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.572149:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547dfd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27291648->27295743] (req 27291648->27295743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.572174:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547dfd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27291648->27295743] (req 27291648->27295743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.572181:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547dfd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27291648->27295743] (req 27291648->27295743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.572578:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495936640:12345-192.168.203.9@tcp:4:dd.0 Request processed in 72909us (73448us total) trans 4294973980 rc 0/0 00000100:00100000:3.0:1789760785.572593:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6692 00000100:00100000:3.0:1789760785.572606:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769753563261 time 769820366412 state 0x2 ref 0 00000100:00100000:0.0:1789760785.575935:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760785.576153:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495936768 00000100:00100000:1.0:1789760785.576167:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495936768 00000100:00100000:1.0:1789760785.576242:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769753563261 time 769820366412 state 0x0 ref 1 00000100:00100000:1.0:1789760785.576261:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769820626828 time 769820366412 state 0x1 ref 1 00000100:00100000:0.0:1789760785.578320:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769820626828 time 769899019757 state 0x0 ref 1 00000100:00100000:0.0:1789760785.578358:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01375c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495936768:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760785.578435:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27295744, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:0.0:1789760785.578457:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547e04 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.578485:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760785.578490:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.578497:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547e04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27295744->27299839] (req 27295744->27299839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.578514:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547e04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27295744->27299839] (req 27295744->27299839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.627256:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973981 is committed 00002000:00010000:2.0:1789760785.627549:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27295744 -> 27299840 00010000:00010000:2.0:1789760785.627570:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547e04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27295744->27299839] (req 27295744->27299839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.627584:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547e04 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27295744->27299839] (req 27295744->27299839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.627595:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547e04 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27295744->27299839] (req 27295744->27299839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.627603:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760785.627610:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547e04 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27295744->27299839] (req 27295744->27299839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.627621:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547e04 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27295744->27299839] (req 27295744->27299839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.627653:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547e04 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27295744->27299839] (req 27295744->27299839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.627662:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547e04 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27295744->27299839] (req 27295744->27299839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.628220:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01375c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495936768:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49871us (52293us total) trans 4294973981 rc 0/0 00000100:00100000:2.0:1789760785.628234:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6693 00000100:00100000:2.0:1789760785.628247:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769820626828 time 769899019757 state 0x2 ref 0 00000100:00100000:1.0:1789760785.634136:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760785.634255:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495936896 00000100:00100000:3.0:1789760785.634269:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495936896 00000100:00100000:3.0:1789760785.634624:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769820626828 time 769899019757 state 0x0 ref 1 00000100:00100000:3.0:1789760785.634644:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769899280173 time 769899019757 state 0x1 ref 1 00000100:00100000:0.0:1789760785.634860:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769899280173 time 769955573129 state 0x0 ref 1 00000100:00100000:0.0:1789760785.634881:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036398c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495936896:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760785.634946:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27299840, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:0.0:1789760785.634959:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547e0b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.634980:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760785.635503:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.635515:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547e0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27299840->27303935] (req 27299840->27303935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.635537:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547e0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27299840->27303935] (req 27299840->27303935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.689573:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973982 is committed 00002000:00010000:1.0:1789760785.690410:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27299840 -> 27303936 00002000:00010000:1.0:1789760785.690420:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760784 -> 1789760785 00002000:00010000:1.0:1789760785.690423:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760784 -> 1789760785 00010000:00010000:1.0:1789760785.690442:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547e0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27299840->27303935] (req 27299840->27303935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.690457:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547e0b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27299840->27303935] (req 27299840->27303935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.690467:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547e0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27299840->27303935] (req 27299840->27303935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.690475:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760785.690482:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547e0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27299840->27303935] (req 27299840->27303935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.690494:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547e0b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27299840->27303935] (req 27299840->27303935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.690522:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547e0b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27299840->27303935] (req 27299840->27303935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.690530:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547e0b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27299840->27303935] (req 27299840->27303935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.694147:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036398c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495936896:12345-192.168.203.9@tcp:4:dd.0 Request processed in 59245us (59996us total) trans 4294973982 rc 0/0 00000100:00100000:1.0:1789760785.694169:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6694 00000100:00100000:1.0:1789760785.694183:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769899280173 time 769955573129 state 0x2 ref 0 00000100:00100000:1.0:1789760785.700060:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760785.700340:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495937024 00000100:00100000:2.0:1789760785.700354:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495937024 00000100:00100000:2.0:1789760785.700428:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769899280173 time 769955573129 state 0x0 ref 1 00000100:00100000:2.0:1789760785.700451:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769955833545 time 769955573129 state 0x1 ref 1 00000100:00100000:2.0:1789760785.700479:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769955833545 time 770021196166 state 0x0 ref 1 00000100:00100000:2.0:1789760785.700494:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495937024:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760785.702237:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27303936, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:2.0:1789760785.702257:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e12 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.702304:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:2.0:1789760785.702310:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.702318:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27303936->27308031] (req 27303936->27308031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.702336:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27303936->27308031] (req 27303936->27308031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.758624:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973983 is committed 00002000:00010000:2.0:1789760785.758874:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27303936 -> 27308032 00010000:00010000:2.0:1789760785.758895:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27303936->27308031] (req 27303936->27308031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.758910:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e12 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27303936->27308031] (req 27303936->27308031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.758920:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27303936->27308031] (req 27303936->27308031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.758934:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760785.758940:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27303936->27308031] (req 27303936->27308031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.758950:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e12 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27303936->27308031] (req 27303936->27308031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.758975:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e12 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27303936->27308031] (req 27303936->27308031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.758984:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e12 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27303936->27308031] (req 27303936->27308031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.759106:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495937024:12345-192.168.203.9@tcp:4:dd.0 Request processed in 58608us (59054us total) trans 4294973983 rc 0/0 00000100:00100000:2.0:1789760785.759119:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6695 00000100:00100000:2.0:1789760785.759133:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769955833545 time 770021196166 state 0x2 ref 0 00000100:00100000:3.0:1789760785.762563:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760785.762974:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495937152 00000100:00100000:2.0:1789760785.762989:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495937152 00000100:00100000:2.0:1789760785.763072:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 769955833545 time 770021196166 state 0x0 ref 1 00000100:00100000:2.0:1789760785.763091:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770021456582 time 770021196166 state 0x1 ref 1 00000100:00100000:2.0:1789760785.763128:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770021456582 time 770083844020 state 0x0 ref 1 00000100:00100000:2.0:1789760785.763147:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495937152:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760785.763216:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27308032, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:2.0:1789760785.763226:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547e19 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.763246:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:2.0:1789760785.763252:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.763260:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547e19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27308032->27312127] (req 27308032->27312127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.763276:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547e19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27308032->27312127] (req 27308032->27312127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760785.797047:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27308032 -> 27312128 00010000:00010000:1.0:1789760785.797077:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547e19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27308032->27312127] (req 27308032->27312127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.797094:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547e19 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27308032->27312127] (req 27308032->27312127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.797105:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547e19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27308032->27312127] (req 27308032->27312127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.797119:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00000001:00080000:0.0:1789760785.797481:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973984 is committed 00010000:00010000:1.0:1789760785.797536:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547e19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27308032->27312127] (req 27308032->27312127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.797551:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547e19 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27308032->27312127] (req 27308032->27312127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.797577:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547e19 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27308032->27312127] (req 27308032->27312127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.797586:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354547e19 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27308032->27312127] (req 27308032->27312127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.797867:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495937152:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34715us (35310us total) trans 4294973984 rc 0/0 00000100:00100000:1.0:1789760785.797880:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6696 00000100:00100000:1.0:1789760785.797892:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770021456582 time 770083844020 state 0x2 ref 0 00000100:00100000:1.0:1789760785.803021:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760785.803208:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495937280 00000100:00100000:0.0:1789760785.803221:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495937280 00000100:00100000:0.0:1789760785.803531:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770021456582 time 770083844020 state 0x0 ref 1 00000100:00100000:0.0:1789760785.803551:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770084104436 time 770083844020 state 0x1 ref 1 00000100:00100000:0.0:1789760785.803644:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770084104436 time 770124359504 state 0x0 ref 1 00000100:00100000:0.0:1789760785.803663:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495937280:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760785.803877:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27312128, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:0.0:1789760785.803889:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547e20 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.803912:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:0.0:1789760785.803918:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.803926:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547e20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27312128->27316223] (req 27312128->27316223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.803944:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547e20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27312128->27316223] (req 27312128->27316223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760785.850478:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27312128 -> 27316224 00010000:00010000:2.0:1789760785.850512:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547e20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27312128->27316223] (req 27312128->27316223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.850530:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547e20 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27312128->27316223] (req 27312128->27316223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.850541:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547e20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27312128->27316223] (req 27312128->27316223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.850550:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760785.850557:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547e20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27312128->27316223] (req 27312128->27316223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.850569:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547e20 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27312128->27316223] (req 27312128->27316223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.850597:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547e20 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27312128->27316223] (req 27312128->27316223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.850606:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547e20 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27312128->27316223] (req 27312128->27316223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.850758:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495937280:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47087us (47739us total) trans 4294973985 rc 0/0 00000100:00100000:2.0:1789760785.850884:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6697 00000100:00100000:2.0:1789760785.850897:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770084104436 time 770124359504 state 0x2 ref 0 00000001:00080000:0.0:1789760785.853359:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973985 is committed 00000100:00100000:0.0:1789760785.855489:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760785.855609:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495937408 00000100:00100000:2.0:1789760785.855623:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495937408 00000100:00100000:2.0:1789760785.855720:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770084104436 time 770124359504 state 0x0 ref 1 00000100:00100000:2.0:1789760785.855741:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770124619920 time 770124359504 state 0x1 ref 1 00000100:00100000:2.0:1789760785.855923:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770124619920 time 770176637750 state 0x0 ref 1 00000100:00100000:2.0:1789760785.855939:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01374a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495937408:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760785.856001:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27316224, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:2.0:1789760785.856012:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e27 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.856044:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:2.0:1789760785.856051:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.856185:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27316224->27320319] (req 27316224->27320319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.856206:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27316224->27320319] (req 27316224->27320319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760785.897228:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27316224 -> 27320320 00010000:00010000:2.0:1789760785.897257:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27316224->27320319] (req 27316224->27320319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.897273:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e27 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27316224->27320319] (req 27316224->27320319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.897321:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e27 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27316224->27320319] (req 27316224->27320319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.897329:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760785.897335:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e27 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27316224->27320319] (req 27316224->27320319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.897345:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e27 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27316224->27320319] (req 27316224->27320319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.897371:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e27 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27316224->27320319] (req 27316224->27320319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.897379:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e27 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27316224->27320319] (req 27316224->27320319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.897489:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01374a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495937408:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41546us (42008us total) trans 4294973986 rc 0/0 00000100:00100000:2.0:1789760785.897502:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6698 00000100:00100000:2.0:1789760785.897515:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770124619920 time 770176637750 state 0x2 ref 0 00000001:00080000:0.0:1789760785.897630:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973986 is committed 00000100:00100000:3.0:1789760785.901117:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760785.901968:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495937536 00000100:00100000:2.0:1789760785.901982:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495937536 00000100:00100000:3.0:1789760785.902203:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770124619920 time 770176637750 state 0x0 ref 1 00000100:00100000:3.0:1789760785.902222:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770176898166 time 770176637750 state 0x1 ref 1 00000100:00100000:3.0:1789760785.902317:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770176898166 time 770223032763 state 0x0 ref 1 00000100:00100000:3.0:1789760785.902333:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495937536:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760785.902401:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27320320, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:3.0:1789760785.902412:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547e2e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.902432:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760785.902438:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.902446:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547e2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27320320->27324415] (req 27320320->27324415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.902461:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547e2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27320320->27324415] (req 27320320->27324415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760785.949856:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27320320 -> 27324416 00010000:00010000:1.0:1789760785.949886:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547e2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27320320->27324415] (req 27320320->27324415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.949903:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547e2e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27320320->27324415] (req 27320320->27324415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.949920:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547e2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27320320->27324415] (req 27320320->27324415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.949929:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760785.949935:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547e2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27320320->27324415] (req 27320320->27324415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.949946:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547e2e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27320320->27324415] (req 27320320->27324415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.949970:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547e2e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27320320->27324415] (req 27320320->27324415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.949978:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354547e2e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27320320->27324415] (req 27320320->27324415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.950105:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495937536:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47770us (48996us total) trans 4294973987 rc 0/0 00000100:00100000:1.0:1789760785.950117:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6699 00000100:00100000:1.0:1789760785.950131:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770176898166 time 770223032763 state 0x2 ref 0 00000001:00080000:0.0:1789760785.950889:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973987 is committed 00000100:00100000:3.0:1789760785.953868:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760785.954034:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495937664 00000100:00100000:2.0:1789760785.954055:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495937664 00000100:00100000:2.0:1789760785.954123:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770176898166 time 770223032763 state 0x0 ref 1 00000100:00100000:2.0:1789760785.954141:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770223293179 time 770223032763 state 0x1 ref 1 00000100:00100000:2.0:1789760785.954171:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770223293179 time 770274888341 state 0x0 ref 1 00000100:00100000:2.0:1789760785.954184:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495937664:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760785.954245:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27324416, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:2.0:1789760785.954256:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e35 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.954276:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:2.0:1789760785.954314:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.954322:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27324416->27328511] (req 27324416->27328511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.954338:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27324416->27328511] (req 27324416->27328511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.990416:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973988 is committed 00002000:00010000:0.0:1789760785.990533:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27324416 -> 27328512 00010000:00010000:0.0:1789760785.990556:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27324416->27328511] (req 27324416->27328511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.990576:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e35 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27324416->27328511] (req 27324416->27328511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.990587:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e35 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27324416->27328511] (req 27324416->27328511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.990595:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760785.990600:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e35 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27324416->27328511] (req 27324416->27328511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.990612:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e35 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27324416->27328511] (req 27324416->27328511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.990636:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e35 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27324416->27328511] (req 27324416->27328511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.990644:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547e35 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27324416->27328511] (req 27324416->27328511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.990948:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495937664:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36757us (37084us total) trans 4294973988 rc 0/0 00000100:00100000:0.0:1789760785.990962:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6700 00000100:00100000:0.0:1789760785.990975:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770223293179 time 770274888341 state 0x2 ref 0 00000100:00100000:3.0:1789760785.994879:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760785.995568:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495937792 00000100:00100000:2.0:1789760785.995584:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495937792 00000100:00100000:2.0:1789760785.995668:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770223293179 time 770274888341 state 0x0 ref 1 00000100:00100000:2.0:1789760785.995827:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770275148757 time 770274888341 state 0x1 ref 1 00000100:00100000:2.0:1789760785.995912:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770275148757 time 770316626702 state 0x0 ref 1 00000100:00100000:2.0:1789760785.995929:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495937792:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760785.995998:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27328512, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:2.0:1789760785.996010:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547e3c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.996042:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:2.0:1789760785.996048:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.996056:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547e3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27328512->27332607] (req 27328512->27332607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.996073:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547e3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27328512->27332607] (req 27328512->27332607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.068727:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973989 is committed 00002000:00010000:0.0:1789760786.068999:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27328512 -> 27332608 00010000:00010000:0.0:1789760786.069020:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547e3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27328512->27332607] (req 27328512->27332607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.069105:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547e3c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27328512->27332607] (req 27328512->27332607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.069116:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547e3c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27328512->27332607] (req 27328512->27332607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.069124:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760786.069130:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547e3c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27328512->27332607] (req 27328512->27332607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.069140:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547e3c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27328512->27332607] (req 27328512->27332607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.069166:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547e3c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27328512->27332607] (req 27328512->27332607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.069175:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354547e3c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27328512->27332607] (req 27328512->27332607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.069458:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495937792:12345-192.168.203.9@tcp:4:dd.0 Request processed in 73525us (74761us total) trans 4294973989 rc 0/0 00000100:00100000:0.0:1789760786.069471:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6701 00000100:00100000:0.0:1789760786.069484:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770275148757 time 770316626702 state 0x2 ref 0 00000100:00100000:0.0:1789760786.075397:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760786.076158:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495937920 00000100:00100000:1.0:1789760786.076174:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495937920 00000100:00100000:1.0:1789760786.076254:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770275148757 time 770316626702 state 0x0 ref 1 00000100:00100000:1.0:1789760786.076274:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770316887118 time 770316626702 state 0x1 ref 1 00000100:00100000:1.0:1789760786.076343:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770316887118 time 770397059158 state 0x0 ref 1 00000100:00100000:1.0:1789760786.076357:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01375180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495937920:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760786.076422:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27332608, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:1.0:1789760786.076433:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547e43 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.076452:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:1.0:1789760786.076458:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.076466:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547e43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27332608->27336703] (req 27332608->27336703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.076482:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547e43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27332608->27336703] (req 27332608->27336703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.135165:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973990 is committed 00002000:00010000:0.0:1789760786.135967:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27332608 -> 27336704 00010000:00010000:0.0:1789760786.135990:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547e43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27332608->27336703] (req 27332608->27336703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.136006:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547e43 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27332608->27336703] (req 27332608->27336703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.136017:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547e43 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27332608->27336703] (req 27332608->27336703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.136026:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760786.136109:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547e43 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27332608->27336703] (req 27332608->27336703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.136122:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547e43 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27332608->27336703] (req 27332608->27336703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.136148:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547e43 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27332608->27336703] (req 27332608->27336703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.136157:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354547e43 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27332608->27336703] (req 27332608->27336703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.136386:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01375180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495937920:12345-192.168.203.9@tcp:4:dd.0 Request processed in 60025us (60997us total) trans 4294973990 rc 0/0 00000100:00100000:0.0:1789760786.136415:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6702 00000100:00100000:0.0:1789760786.136429:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770316887118 time 770397059158 state 0x2 ref 0 00000100:00100000:3.0:1789760786.144263:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760786.144942:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495938048 00000100:00100000:0.0:1789760786.144956:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495938048 00000100:00100000:0.0:1789760786.145040:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770316887118 time 770397059158 state 0x0 ref 1 00000100:00100000:0.0:1789760786.145061:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770397319574 time 770397059158 state 0x1 ref 1 00000100:00100000:0.0:1789760786.145090:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770397319574 time 770465806987 state 0x0 ref 1 00000100:00100000:0.0:1789760786.145106:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495938048:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760786.145174:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27336704, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:0.0:1789760786.145185:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e4a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.145205:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:0.0:1789760786.145210:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.145218:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27336704->27340799] (req 27336704->27340799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.145234:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27336704->27340799] (req 27336704->27340799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.196260:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973991 is committed 00002000:00010000:0.0:1789760786.196394:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27336704 -> 27340800 00010000:00010000:0.0:1789760786.196416:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27336704->27340799] (req 27336704->27340799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.196432:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e4a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27336704->27340799] (req 27336704->27340799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.196441:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e4a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27336704->27340799] (req 27336704->27340799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.196449:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ae00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760786.196454:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e4a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27336704->27340799] (req 27336704->27340799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.196464:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e4a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27336704->27340799] (req 27336704->27340799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.196490:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e4a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27336704->27340799] (req 27336704->27340799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.196497:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e4a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27336704->27340799] (req 27336704->27340799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.196670:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495938048:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51560us (52414us total) trans 4294973991 rc 0/0 00000100:00100000:0.0:1789760786.196697:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6703 00000100:00100000:0.0:1789760786.196721:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770397319574 time 770465806987 state 0x2 ref 0 00000100:00100000:3.0:1789760786.199122:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760786.199178:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495938176 00000100:00100000:0.0:1789760786.199191:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495938176 00000100:00100000:0.0:1789760786.199256:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770397319574 time 770465806987 state 0x0 ref 1 00000100:00100000:0.0:1789760786.199270:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770466067403 time 770465806987 state 0x1 ref 1 00000100:00100000:3.0:1789760786.199326:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770466067403 time 770520039683 state 0x0 ref 1 00000100:00100000:2.0:1789760786.199454:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495938176:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760786.199517:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27340800, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:2.0:1789760786.199526:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547e51 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.199544:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:2.0:1789760786.199549:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.199556:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547e51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27340800->27344895] (req 27340800->27344895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.199570:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547e51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27340800->27344895] (req 27340800->27344895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760786.245410:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27340800 -> 27344896 00010000:00010000:3.0:1789760786.245436:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547e51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27340800->27344895] (req 27340800->27344895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.245453:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547e51 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27340800->27344895] (req 27340800->27344895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.245463:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547e51 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27340800->27344895] (req 27340800->27344895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.245470:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760786.245477:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547e51 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27340800->27344895] (req 27340800->27344895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.245486:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547e51 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27340800->27344895] (req 27340800->27344895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.245511:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547e51 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27340800->27344895] (req 27340800->27344895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.245519:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x4936748354547e51 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27340800->27344895] (req 27340800->27344895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.245650:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495938176:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46243us (46529us total) trans 4294973992 rc 0/0 00000100:00100000:3.0:1789760786.245662:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6704 00000100:00100000:3.0:1789760786.245674:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770466067403 time 770520039683 state 0x2 ref 0 00000100:00100000:2.0:1789760786.249491:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760786.249617:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495938304 00000100:00100000:3.0:1789760786.249630:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495938304 00000100:00100000:3.0:1789760786.249722:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770466067403 time 770520039683 state 0x0 ref 1 00000100:00100000:3.0:1789760786.249743:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770520300099 time 770520039683 state 0x1 ref 1 00000100:00100000:3.0:1789760786.249768:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770520300099 time 770570484630 state 0x0 ref 1 00000100:00100000:3.0:1789760786.249782:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495938304:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760786.249847:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27344896, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:3.0:1789760786.249859:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e58 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.249880:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760786.249885:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.249894:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27344896->27348991] (req 27344896->27348991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.249911:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27344896->27348991] (req 27344896->27348991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.250942:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973992 is committed 00010000:00010000:1.0:1789760786.271778:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760786.290376:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973993 is committed 00002000:00010000:3.0:1789760786.290488:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27344896 -> 27348992 00010000:00010000:3.0:1789760786.290509:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27344896->27348991] (req 27344896->27348991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.290524:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e58 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27344896->27348991] (req 27344896->27348991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.290535:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27344896->27348991] (req 27344896->27348991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.290543:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760786.290549:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27344896->27348991] (req 27344896->27348991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.290560:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e58 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27344896->27348991] (req 27344896->27348991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.290590:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e58 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27344896->27348991] (req 27344896->27348991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.290598:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e58 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27344896->27348991] (req 27344896->27348991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.290945:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495938304:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41157us (41459us total) trans 4294973993 rc 0/0 00000100:00100000:3.0:1789760786.290958:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6705 00000100:00100000:3.0:1789760786.290971:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770520300099 time 770570484630 state 0x2 ref 0 00000100:00100000:2.0:1789760786.294647:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760786.294836:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495938432 00000100:00100000:0.0:1789760786.294850:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495938432 00000100:00100000:0.0:1789760786.294930:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770520300099 time 770570484630 state 0x0 ref 1 00000100:00100000:0.0:1789760786.294948:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770570745046 time 770570484630 state 0x1 ref 1 00000100:00100000:0.0:1789760786.295023:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770570745046 time 770615739163 state 0x0 ref 1 00000100:00100000:0.0:1789760786.295172:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495938432:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760786.295237:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27348992, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:0.0:1789760786.295248:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547e5f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.295269:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:0.0:1789760786.295274:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.295462:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547e5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27348992->27353087] (req 27348992->27353087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.295482:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547e5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27348992->27353087] (req 27348992->27353087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760786.333761:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27348992 -> 27353088 00010000:00010000:0.0:1789760786.333789:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547e5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27348992->27353087] (req 27348992->27353087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.333803:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547e5f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27348992->27353087] (req 27348992->27353087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.333813:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547e5f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27348992->27353087] (req 27348992->27353087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.333820:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94aa00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760786.333826:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547e5f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27348992->27353087] (req 27348992->27353087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.333835:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547e5f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27348992->27353087] (req 27348992->27353087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.333861:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547e5f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27348992->27353087] (req 27348992->27353087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.333869:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547e5f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27348992->27353087] (req 27348992->27353087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.333989:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495938432:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38816us (39351us total) trans 4294973994 rc 0/0 00000100:00100000:0.0:1789760786.334001:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6706 00000100:00100000:0.0:1789760786.334013:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770570745046 time 770615739163 state 0x2 ref 0 00000001:00080000:3.0:1789760786.334764:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973994 is committed 00000100:00100000:0.0:1789760786.337938:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760786.338082:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495938560 00000100:00100000:3.0:1789760786.338095:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495938560 00000100:00100000:3.0:1789760786.338169:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770570745046 time 770615739163 state 0x0 ref 1 00000100:00100000:3.0:1789760786.338187:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770615999579 time 770615739163 state 0x1 ref 1 00000100:00100000:3.0:1789760786.338223:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770615999579 time 770658939141 state 0x0 ref 1 00000100:00100000:3.0:1789760786.338237:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01376300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495938560:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760786.338336:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27353088, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:3.0:1789760786.338348:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e66 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.338375:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760786.338380:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.338388:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27353088->27357183] (req 27353088->27357183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.338404:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27353088->27357183] (req 27353088->27357183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760786.367336:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27353088 -> 27357184 00010000:00010000:3.0:1789760786.367362:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27353088->27357183] (req 27353088->27357183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.367377:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e66 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27353088->27357183] (req 27353088->27357183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.367386:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e66 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27353088->27357183] (req 27353088->27357183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.367395:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760786.367401:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e66 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27353088->27357183] (req 27353088->27357183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.367411:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e66 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27353088->27357183] (req 27353088->27357183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.367435:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e66 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27353088->27357183] (req 27353088->27357183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.367444:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e66 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27353088->27357183] (req 27353088->27357183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.367640:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01376300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495938560:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29397us (29708us total) trans 4294973995 rc 0/0 00000100:00100000:3.0:1789760786.367654:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6707 00000100:00100000:3.0:1789760786.367666:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770615999579 time 770658939141 state 0x2 ref 0 00000001:00080000:3.0:1789760786.367733:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973995 is committed 00000100:00100000:2.0:1789760786.372090:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760786.372193:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495938688 00000100:00100000:3.0:1789760786.372205:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495938688 00000100:00100000:3.0:1789760786.372274:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770615999579 time 770658939141 state 0x0 ref 1 00000100:00100000:3.0:1789760786.372323:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770659199557 time 770658939141 state 0x1 ref 1 00000100:00100000:3.0:1789760786.372354:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770659199557 time 770693070924 state 0x0 ref 1 00000100:00100000:3.0:1789760786.372368:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495938688:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760786.372432:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27357184, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:3.0:1789760786.372442:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547e6d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.372462:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760786.372468:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.372475:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547e6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27357184->27361279] (req 27357184->27361279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.372492:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547e6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27357184->27361279] (req 27357184->27361279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.405574:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973996 is committed 00002000:00010000:3.0:1789760786.405825:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27357184 -> 27361280 00010000:00010000:3.0:1789760786.405846:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547e6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27357184->27361279] (req 27357184->27361279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.405862:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547e6d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27357184->27361279] (req 27357184->27361279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.405873:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547e6d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27357184->27361279] (req 27357184->27361279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.405881:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760786.405887:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547e6d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27357184->27361279] (req 27357184->27361279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.405898:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547e6d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27357184->27361279] (req 27357184->27361279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.405922:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547e6d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27357184->27361279] (req 27357184->27361279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.405931:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547e6d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27357184->27361279] (req 27357184->27361279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.406160:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495938688:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33786us (34079us total) trans 4294973996 rc 0/0 00000100:00100000:3.0:1789760786.406174:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6708 00000100:00100000:3.0:1789760786.406187:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770659199557 time 770693070924 state 0x2 ref 0 00000100:00100000:2.0:1789760786.410006:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760786.410214:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495938816 00000100:00100000:0.0:1789760786.410228:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495938816 00000100:00100000:0.0:1789760786.414381:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770659199557 time 770693070924 state 0x0 ref 1 00000100:00100000:0.0:1789760786.414404:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770693331340 time 770693070924 state 0x1 ref 1 00000100:00100000:3.0:1789760786.414583:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770693331340 time 770735295969 state 0x0 ref 1 00000100:00100000:3.0:1789760786.414614:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495938816:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760786.414695:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27361280, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:3.0:1789760786.414715:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e74 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.414735:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760786.414740:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.414747:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27361280->27365375] (req 27361280->27365375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.414885:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27361280->27365375] (req 27361280->27365375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760786.453565:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27361280 -> 27365376 00010000:00010000:3.0:1789760786.453597:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27361280->27365375] (req 27361280->27365375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.453614:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e74 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27361280->27365375] (req 27361280->27365375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.453625:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e74 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27361280->27365375] (req 27361280->27365375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.453633:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760786.453639:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e74 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27361280->27365375] (req 27361280->27365375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.453651:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e74 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27361280->27365375] (req 27361280->27365375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.453679:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e74 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27361280->27365375] (req 27361280->27365375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.453700:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e74 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27361280->27365375] (req 27361280->27365375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.453861:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495938816:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39256us (43863us total) trans 4294973997 rc 0/0 00000100:00100000:3.0:1789760786.453874:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6709 00000100:00100000:3.0:1789760786.453886:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770693331340 time 770735295969 state 0x2 ref 0 00000001:00080000:0.0:1789760786.457176:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973997 is committed 00000100:00100000:2.0:1789760786.457395:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760786.457496:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495938944 00000100:00100000:3.0:1789760786.457508:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495938944 00000100:00100000:3.0:1789760786.457576:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770693331340 time 770735295969 state 0x0 ref 1 00000100:00100000:3.0:1789760786.457595:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770735556385 time 770735295969 state 0x1 ref 1 00000100:00100000:3.0:1789760786.457720:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770735556385 time 770778435354 state 0x0 ref 1 00000100:00100000:3.0:1789760786.457735:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495938944:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760786.457811:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27365376, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:3.0:1789760786.457821:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547e7b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.457840:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760786.457845:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.457854:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547e7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27365376->27369471] (req 27365376->27369471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.457871:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547e7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27365376->27369471] (req 27365376->27369471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.504129:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973998 is committed 00002000:00010000:0.0:1789760786.504353:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27365376 -> 27369472 00010000:00010000:0.0:1789760786.504372:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547e7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27365376->27369471] (req 27365376->27369471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.504386:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547e7b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27365376->27369471] (req 27365376->27369471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.504397:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547e7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27365376->27369471] (req 27365376->27369471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.504405:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760786.504411:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547e7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27365376->27369471] (req 27365376->27369471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.504422:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547e7b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27365376->27369471] (req 27365376->27369471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.504444:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547e7b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27365376->27369471] (req 27365376->27369471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.504454:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354547e7b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27365376->27369471] (req 27365376->27369471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.504577:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495938944:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46838us (47183us total) trans 4294973998 rc 0/0 00000100:00100000:0.0:1789760786.504590:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6710 00000100:00100000:0.0:1789760786.504603:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770735556385 time 770778435354 state 0x2 ref 0 00000100:00100000:2.0:1789760786.508240:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760786.508489:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495939072 00000100:00100000:3.0:1789760786.508502:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495939072 00000100:00100000:3.0:1789760786.508575:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770735556385 time 770778435354 state 0x0 ref 1 00000100:00100000:3.0:1789760786.508595:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770778695770 time 770778435354 state 0x1 ref 1 00000100:00100000:3.0:1789760786.508666:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770778695770 time 770829382280 state 0x0 ref 1 00000100:00100000:3.0:1789760786.508681:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04931f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495939072:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760786.508771:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27369472, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:3.0:1789760786.508782:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e82 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.508801:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760786.508806:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.508814:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27369472->27373567] (req 27369472->27373567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.508829:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27369472->27373567] (req 27369472->27373567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.563530:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973999 is committed 00002000:00010000:1.0:1789760786.563654:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27369472 -> 27373568 00010000:00010000:1.0:1789760786.563679:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27369472->27373567] (req 27369472->27373567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.563835:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e82 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27369472->27373567] (req 27369472->27373567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.563846:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e82 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27369472->27373567] (req 27369472->27373567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.563860:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760786.563867:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e82 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27369472->27373567] (req 27369472->27373567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.563877:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e82 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27369472->27373567] (req 27369472->27373567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.563903:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e82 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27369472->27373567] (req 27369472->27373567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.563912:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547e82 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27369472->27373567] (req 27369472->27373567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.564159:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495939072:12345-192.168.203.9@tcp:4:dd.0 Request processed in 55471us (55924us total) trans 4294973999 rc 0/0 00000100:00100000:1.0:1789760786.564172:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6711 00000100:00100000:1.0:1789760786.564187:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770778695770 time 770829382280 state 0x2 ref 0 00000100:00100000:2.0:1789760786.571539:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760786.571736:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495939200 00000100:00100000:0.0:1789760786.571749:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495939200 00000100:00100000:0.0:1789760786.571832:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770778695770 time 770829382280 state 0x0 ref 1 00000100:00100000:0.0:1789760786.571860:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770829642696 time 770829382280 state 0x1 ref 1 00000100:00100000:0.0:1789760786.571891:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770829642696 time 770892608194 state 0x0 ref 1 00000100:00100000:0.0:1789760786.571905:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495939200:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760786.571972:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27373568, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:0.0:1789760786.571982:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547e89 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.572001:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:0.0:1789760786.572007:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.572015:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547e89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27373568->27377663] (req 27373568->27377663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.572048:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547e89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27373568->27377663] (req 27373568->27377663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.627875:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974000 is committed 00002000:00010000:3.0:1789760786.628021:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27373568 -> 27377664 00010000:00010000:3.0:1789760786.628113:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547e89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27373568->27377663] (req 27373568->27377663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.628129:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547e89 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27373568->27377663] (req 27373568->27377663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.628139:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547e89 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27373568->27377663] (req 27373568->27377663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.628147:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94aa00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760786.628153:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547e89 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27373568->27377663] (req 27373568->27377663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.628163:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547e89 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27373568->27377663] (req 27373568->27377663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.628194:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547e89 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27373568->27377663] (req 27373568->27377663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.628203:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354547e89 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27373568->27377663] (req 27373568->27377663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.628582:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495939200:12345-192.168.203.9@tcp:4:dd.0 Request processed in 56661us (57041us total) trans 4294974000 rc 0/0 00000100:00100000:0.0:1789760786.628597:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6712 00000100:00100000:0.0:1789760786.628610:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770829642696 time 770892608194 state 0x2 ref 0 00000100:00100000:2.0:1789760786.633367:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760786.633489:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495939328 00000100:00100000:0.0:1789760786.633506:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495939328 00000100:00100000:0.0:1789760786.633942:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770829642696 time 770892608194 state 0x0 ref 1 00000100:00100000:0.0:1789760786.633962:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770892868610 time 770892608194 state 0x1 ref 1 00000100:00100000:0.0:1789760786.634050:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770892868610 time 770954764640 state 0x0 ref 1 00000100:00100000:0.0:1789760786.634067:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04930380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495939328:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760786.634134:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27377664, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:0.0:1789760786.634146:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e90 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.634166:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:0.0:1789760786.634172:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.634179:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27377664->27381759] (req 27377664->27381759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.634201:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27377664->27381759] (req 27377664->27381759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.667621:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974001 is committed 00002000:00010000:0.0:1789760786.667897:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27377664 -> 27381760 00002000:00010000:0.0:1789760786.667903:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760785 -> 1789760786 00002000:00010000:0.0:1789760786.667906:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760785 -> 1789760786 00010000:00010000:0.0:1789760786.667924:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27377664->27381759] (req 27377664->27381759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.667939:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e90 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27377664->27381759] (req 27377664->27381759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.667949:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27377664->27381759] (req 27377664->27381759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.667957:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ae00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760786.667963:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27377664->27381759] (req 27377664->27381759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.667974:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e90 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27377664->27381759] (req 27377664->27381759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.667999:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e90 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27377664->27381759] (req 27377664->27381759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.668011:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e90 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27377664->27381759] (req 27377664->27381759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.668251:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04930380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495939328:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34181us (34892us total) trans 4294974001 rc 0/0 00000100:00100000:0.0:1789760786.668263:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6713 00000100:00100000:0.0:1789760786.668332:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770892868610 time 770954764640 state 0x2 ref 0 00000100:00100000:2.0:1789760786.678516:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760786.678612:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495939456 00000100:00100000:1.0:1789760786.678625:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495939456 00000100:00100000:1.0:1789760786.678832:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770892868610 time 770954764640 state 0x0 ref 1 00000100:00100000:1.0:1789760786.678852:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770955025056 time 770954764640 state 0x1 ref 1 00000100:00100000:1.0:1789760786.678885:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770955025056 time 770999601514 state 0x0 ref 1 00000100:00100000:1.0:1789760786.678900:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04932d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495939456:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760786.678966:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27381760, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:1.0:1789760786.678978:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354547e97 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.678999:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:1.0:1789760786.679004:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.679012:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354547e97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27381760->27385855] (req 27381760->27385855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.679141:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354547e97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27381760->27385855] (req 27381760->27385855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.718789:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f04293b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520316160:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760786.718884:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760786.719131:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520316160 00000100:00100000:1.0:1789760786.719142:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520316160 00000100:00080000:2.0:1789760786.719185:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760786.719209:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 217 00000100:00100000:1.0:1789760786.719221:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520316160:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760786.719264:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528023552 pending 1703936 free 3830448128 avail 3828350976 00000100:00100000:1.0:1789760786.719346:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520316160:12345-0@lo:13:osp-pre-0-0.0 Request processed in 127us (461us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760786.719354:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 217 00000100:00100000:3.0:1789760786.719521:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f04293b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520316160:0@lo:13:osp-pre-0-0.0 00002000:00010000:3.0:1789760786.721502:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27381760 -> 27385856 00010000:00010000:3.0:1789760786.721526:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354547e97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27381760->27385855] (req 27381760->27385855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.721543:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354547e97 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27381760->27385855] (req 27381760->27385855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.721553:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354547e97 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27381760->27385855] (req 27381760->27385855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.721561:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760786.721567:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354547e97 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27381760->27385855] (req 27381760->27385855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.721576:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354547e97 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27381760->27385855] (req 27381760->27385855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.721605:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354547e97 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27381760->27385855] (req 27381760->27385855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.721613:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354547e97 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27381760->27385855] (req 27381760->27385855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.721748:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04932d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495939456:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42844us (43239us total) trans 4294974002 rc 0/0 00000100:00100000:3.0:1789760786.721827:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6714 00000100:00100000:3.0:1789760786.721840:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770955025056 time 770999601514 state 0x2 ref 0 00000001:00080000:3.0:1789760786.721887:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974002 is committed 00000100:00100000:2.0:1789760786.726246:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760786.726599:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495939584 00000100:00100000:0.0:1789760786.726614:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495939584 00000100:00100000:0.0:1789760786.726824:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770955025056 time 770999601514 state 0x0 ref 1 00000100:00100000:0.0:1789760786.726846:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770999861930 time 770999601514 state 0x1 ref 1 00000100:00100000:0.0:1789760786.726919:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770999861930 time 771047635486 state 0x0 ref 1 00000100:00100000:0.0:1789760786.726935:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04933100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495939584:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760786.727002:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27385856, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:0.0:1789760786.727013:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e9e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.727042:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760786.727048:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.727055:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27385856->27389951] (req 27385856->27389951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.727077:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27385856->27389951] (req 27385856->27389951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.781853:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974003 is committed 00002000:00010000:2.0:1789760786.782219:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27385856 -> 27389952 00010000:00010000:2.0:1789760786.782241:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27385856->27389951] (req 27385856->27389951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.782258:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e9e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27385856->27389951] (req 27385856->27389951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.782269:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e9e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27385856->27389951] (req 27385856->27389951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.782309:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ae00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760786.782317:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e9e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27385856->27389951] (req 27385856->27389951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.782327:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e9e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27385856->27389951] (req 27385856->27389951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.782352:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e9e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27385856->27389951] (req 27385856->27389951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.782360:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354547e9e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27385856->27389951] (req 27385856->27389951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.782485:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04933100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495939584:12345-192.168.203.9@tcp:4:dd.0 Request processed in 55546us (56248us total) trans 4294974003 rc 0/0 00000100:00100000:2.0:1789760786.782501:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6715 00000100:00100000:2.0:1789760786.782513:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770999861930 time 771047635486 state 0x2 ref 0 00000100:00100000:1.0:1789760786.786622:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760786.786911:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495939712 00000100:00100000:3.0:1789760786.786925:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495939712 00000100:00100000:3.0:1789760786.787014:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 770999861930 time 771047635486 state 0x0 ref 1 00000100:00100000:3.0:1789760786.787044:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771047895902 time 771047635486 state 0x1 ref 1 00000100:00100000:3.0:1789760786.787076:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771047895902 time 771107792739 state 0x0 ref 1 00000100:00100000:3.0:1789760786.787091:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495939712:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760786.787162:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27389952, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:3.0:1789760786.787174:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547ea5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.787200:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760786.787206:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.787214:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547ea5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27389952->27394047] (req 27389952->27394047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.787231:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547ea5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27389952->27394047] (req 27389952->27394047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.841515:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974004 is committed 00002000:00010000:0.0:1789760786.841736:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27389952 -> 27394048 00010000:00010000:0.0:1789760786.841882:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547ea5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27389952->27394047] (req 27389952->27394047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.841899:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547ea5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27389952->27394047] (req 27389952->27394047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.841909:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547ea5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27389952->27394047] (req 27389952->27394047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.841917:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760786.841923:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547ea5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27389952->27394047] (req 27389952->27394047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.841935:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547ea5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27389952->27394047] (req 27389952->27394047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.841963:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547ea5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27389952->27394047] (req 27389952->27394047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.841972:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547ea5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27389952->27394047] (req 27389952->27394047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.842193:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495939712:12345-192.168.203.9@tcp:4:dd.0 Request processed in 55097us (55578us total) trans 4294974004 rc 0/0 00000100:00100000:0.0:1789760786.842204:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6716 00000100:00100000:0.0:1789760786.842217:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771047895902 time 771107792739 state 0x2 ref 0 00000100:00100000:3.0:1789760786.846256:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760786.846528:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495939840 00000100:00100000:1.0:1789760786.846552:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495939840 00000100:00100000:1.0:1789760786.846629:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771047895902 time 771107792739 state 0x0 ref 1 00000100:00100000:1.0:1789760786.846653:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771108053155 time 771107792739 state 0x1 ref 1 00000100:00100000:0.0:1789760786.846887:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771108053155 time 771167599225 state 0x0 ref 1 00000100:00100000:0.0:1789760786.846905:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495939840:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760786.846972:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27394048, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:0.0:1789760786.846982:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547eac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.847002:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760786.847008:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.847017:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547eac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27394048->27398143] (req 27394048->27398143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.847109:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547eac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27394048->27398143] (req 27394048->27398143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.888332:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974005 is committed 00002000:00010000:0.0:1789760786.888656:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27394048 -> 27398144 00010000:00010000:0.0:1789760786.888676:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547eac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27394048->27398143] (req 27394048->27398143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.888823:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547eac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27394048->27398143] (req 27394048->27398143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.888834:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547eac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27394048->27398143] (req 27394048->27398143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.888842:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760786.888848:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547eac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27394048->27398143] (req 27394048->27398143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.888859:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547eac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27394048->27398143] (req 27394048->27398143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.888883:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547eac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27394048->27398143] (req 27394048->27398143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.888890:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354547eac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27394048->27398143] (req 27394048->27398143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.889274:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495939840:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42367us (43026us total) trans 4294974005 rc 0/0 00000100:00100000:0.0:1789760786.889337:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6717 00000100:00100000:0.0:1789760786.889350:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771108053155 time 771167599225 state 0x2 ref 0 00000100:00100000:1.0:1789760786.894583:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760786.894842:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495939968 00000100:00100000:2.0:1789760786.894856:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495939968 00000100:00100000:2.0:1789760786.894935:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771108053155 time 771167599225 state 0x0 ref 1 00000100:00100000:2.0:1789760786.894956:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771167859641 time 771167599225 state 0x1 ref 1 00000100:00100000:2.0:1789760786.895050:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771167859641 time 771215759409 state 0x0 ref 1 00000100:00100000:2.0:1789760786.895064:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495939968:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760786.895137:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27398144, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:2.0:1789760786.895148:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547eb3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.895168:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:2.0:1789760786.895174:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.895182:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547eb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27398144->27402239] (req 27398144->27402239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.895199:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547eb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27398144->27402239] (req 27398144->27402239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760786.935525:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27398144 -> 27402240 00010000:00010000:2.0:1789760786.935551:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547eb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27398144->27402239] (req 27398144->27402239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.935568:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547eb3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27398144->27402239] (req 27398144->27402239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.935579:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547eb3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27398144->27402239] (req 27398144->27402239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.935588:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760786.935603:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547eb3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27398144->27402239] (req 27398144->27402239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.935614:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547eb3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27398144->27402239] (req 27398144->27402239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.935638:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547eb3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27398144->27402239] (req 27398144->27402239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.935645:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547eb3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27398144->27402239] (req 27398144->27402239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.936671:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495939968:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41599us (42093us total) trans 4294974006 rc 0/0 00000100:00100000:2.0:1789760786.936692:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6718 00000100:00100000:2.0:1789760786.936712:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771167859641 time 771215759409 state 0x2 ref 0 00000001:00080000:2.0:1789760786.937030:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974006 is committed 00000100:00100000:1.0:1789760786.939778:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760786.939979:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495940096 00000100:00100000:0.0:1789760786.939993:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495940096 00000100:00100000:0.0:1789760786.940083:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771167859641 time 771215759409 state 0x0 ref 1 00000100:00100000:0.0:1789760786.940103:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771216019825 time 771215759409 state 0x1 ref 1 00000100:00100000:0.0:1789760786.940141:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771216019825 time 771260857729 state 0x0 ref 1 00000100:00100000:0.0:1789760786.940157:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495940096:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760786.940226:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27402240, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:0.0:1789760786.940236:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547eba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.940257:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760786.940263:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.940271:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547eba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27402240->27406335] (req 27402240->27406335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.940332:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547eba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27402240->27406335] (req 27402240->27406335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.988660:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974007 is committed 00002000:00010000:0.0:1789760786.988989:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27402240 -> 27406336 00010000:00010000:0.0:1789760786.989010:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547eba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27402240->27406335] (req 27402240->27406335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.989026:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547eba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27402240->27406335] (req 27402240->27406335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.989037:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547eba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27402240->27406335] (req 27402240->27406335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.989046:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760786.989053:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547eba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27402240->27406335] (req 27402240->27406335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.989063:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547eba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27402240->27406335] (req 27402240->27406335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.989151:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547eba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27402240->27406335] (req 27402240->27406335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.989161:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547eba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27402240->27406335] (req 27402240->27406335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.989405:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495940096:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49243us (49634us total) trans 4294974007 rc 0/0 00000100:00100000:0.0:1789760786.989417:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6719 00000100:00100000:0.0:1789760786.989431:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771216019825 time 771260857729 state 0x2 ref 0 00000100:00100000:0.0:1789760786.992790:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760786.992946:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495940224 00000100:00100000:2.0:1789760786.992962:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495940224 00000100:00100000:2.0:1789760786.993038:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771216019825 time 771260857729 state 0x0 ref 1 00000100:00100000:2.0:1789760786.993057:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771261118145 time 771260857729 state 0x1 ref 1 00000100:00100000:2.0:1789760786.993101:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771261118145 time 771313817422 state 0x0 ref 1 00000100:00100000:2.0:1789760786.993116:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01374e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495940224:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760786.993251:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27406336, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:2.0:1789760786.993264:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ec1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.993315:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:2.0:1789760786.993321:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.993330:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ec1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27406336->27410431] (req 27406336->27410431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.993346:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ec1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27406336->27410431] (req 27406336->27410431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.054465:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974008 is committed 00002000:00010000:3.0:1789760787.054613:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27406336 -> 27410432 00010000:00010000:3.0:1789760787.054634:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ec1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27406336->27410431] (req 27406336->27410431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.054651:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ec1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27406336->27410431] (req 27406336->27410431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.054661:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ec1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27406336->27410431] (req 27406336->27410431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.054670:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760787.054676:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ec1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27406336->27410431] (req 27406336->27410431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.054697:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ec1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27406336->27410431] (req 27406336->27410431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.054731:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ec1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27406336->27410431] (req 27406336->27410431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.054749:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ec1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27406336->27410431] (req 27406336->27410431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.054867:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01374e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495940224:12345-192.168.203.9@tcp:4:dd.0 Request processed in 61747us (62085us total) trans 4294974008 rc 0/0 00000100:00100000:3.0:1789760787.054879:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6720 00000100:00100000:3.0:1789760787.054891:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771261118145 time 771313817422 state 0x2 ref 0 00000100:00100000:2.0:1789760787.068496:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760787.068759:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495940352 00000100:00100000:0.0:1789760787.068772:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495940352 00000100:00080000:2.0:1789760787.068861:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760787.068872:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771261118145 time 771313817422 state 0x0 ref 1 00000100:00100000:0.0:1789760787.068892:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771314077838 time 771313817422 state 0x1 ref 1 00000100:00100000:0.0:1789760787.068932:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771314077838 time 771389647491 state 0x0 ref 1 00000100:00100000:0.0:1789760787.068948:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04933480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495940352:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760787.069018:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27410432, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:0.0:1789760787.069029:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ec8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.069049:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760787.069054:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.069061:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ec8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27410432->27414527] (req 27410432->27414527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.069076:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ec8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27410432->27414527] (req 27410432->27414527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760787.127419:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27410432 -> 27414528 00010000:00010000:0.0:1789760787.127449:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ec8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27410432->27414527] (req 27410432->27414527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.127466:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ec8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27410432->27414527] (req 27410432->27414527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.127476:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ec8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27410432->27414527] (req 27410432->27414527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.127485:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760787.127491:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ec8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27410432->27414527] (req 27410432->27414527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.127503:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ec8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27410432->27414527] (req 27410432->27414527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.127516:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974009 is committed 00010000:00010000:0.0:1789760787.127530:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ec8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27410432->27414527] (req 27410432->27414527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.127537:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ec8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27410432->27414527] (req 27410432->27414527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.127658:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04933480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495940352:12345-192.168.203.9@tcp:4:dd.0 Request processed in 58707us (59170us total) trans 4294974009 rc 0/0 00000100:00100000:0.0:1789760787.127670:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6721 00000100:00100000:0.0:1789760787.127682:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771314077838 time 771389647491 state 0x2 ref 0 00000100:00100000:2.0:1789760787.130885:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760787.130958:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495940480 00000100:00100000:0.0:1789760787.130971:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495940480 00000100:00100000:0.0:1789760787.131048:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771314077838 time 771389647491 state 0x0 ref 1 00000100:00100000:0.0:1789760787.131067:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771389907907 time 771389647491 state 0x1 ref 1 00000100:00100000:0.0:1789760787.131106:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771389907907 time 771451822689 state 0x0 ref 1 00000100:00100000:0.0:1789760787.131120:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04932680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495940480:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760787.131185:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27414528, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:0.0:1789760787.131196:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547ecf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.131217:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760787.131224:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.131232:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547ecf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27414528->27418623] (req 27414528->27418623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.131248:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547ecf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27414528->27418623] (req 27414528->27418623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.173973:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974010 is committed 00002000:00010000:0.0:1789760787.174162:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27414528 -> 27418624 00010000:00010000:0.0:1789760787.174187:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547ecf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27414528->27418623] (req 27414528->27418623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.174202:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547ecf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27414528->27418623] (req 27414528->27418623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.174212:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547ecf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27414528->27418623] (req 27414528->27418623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.174221:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760787.174227:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547ecf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27414528->27418623] (req 27414528->27418623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.174238:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547ecf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27414528->27418623] (req 27414528->27418623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.174265:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547ecf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27414528->27418623] (req 27414528->27418623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.174272:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547ecf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27414528->27418623] (req 27414528->27418623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.174541:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04932680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495940480:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43299us (43544us total) trans 4294974010 rc 0/0 00000100:00100000:0.0:1789760787.174554:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6722 00000100:00100000:0.0:1789760787.178096:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771389907907 time 771451822689 state 0x2 ref 0 00000100:00100000:3.0:1789760787.178487:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760787.179815:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495940608 00000100:00100000:2.0:1789760787.179834:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495940608 00000100:00100000:2.0:1789760787.179907:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771389907907 time 771451822689 state 0x0 ref 1 00000100:00100000:2.0:1789760787.179924:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771452083105 time 771451822689 state 0x1 ref 1 00000100:00100000:2.0:1789760787.179957:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771452083105 time 771500674832 state 0x0 ref 1 00000100:00100000:2.0:1789760787.179971:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495940608:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760787.180020:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27418624, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:2.0:1789760787.180029:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547ed6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.180046:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:2.0:1789760787.180051:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.180057:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547ed6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27418624->27422719] (req 27418624->27422719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.180071:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547ed6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27418624->27422719] (req 27418624->27422719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.241948:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974011 is committed 00002000:00010000:3.0:1789760787.242097:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27418624 -> 27422720 00010000:00010000:3.0:1789760787.242118:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547ed6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27418624->27422719] (req 27418624->27422719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.242134:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547ed6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27418624->27422719] (req 27418624->27422719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.242144:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547ed6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27418624->27422719] (req 27418624->27422719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.242153:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760787.242160:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547ed6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27418624->27422719] (req 27418624->27422719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.242171:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547ed6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27418624->27422719] (req 27418624->27422719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.242270:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547ed6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27418624->27422719] (req 27418624->27422719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.242315:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547ed6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27418624->27422719] (req 27418624->27422719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.242447:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495940608:12345-192.168.203.9@tcp:4:dd.0 Request processed in 62472us (63956us total) trans 4294974011 rc 0/0 00000100:00100000:3.0:1789760787.242461:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6723 00000100:00100000:3.0:1789760787.242474:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771452083105 time 771500674832 state 0x2 ref 0 00000100:00100000:2.0:1789760787.254672:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760787.255952:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495940736 00000100:00100000:0.0:1789760787.255974:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495940736 00000100:00100000:0.0:1789760787.256066:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771452083105 time 771500674832 state 0x0 ref 1 00000100:00100000:0.0:1789760787.256086:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771500935248 time 771500674832 state 0x1 ref 1 00000100:00100000:0.0:1789760787.256130:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771500935248 time 771576845391 state 0x0 ref 1 00000100:00100000:0.0:1789760787.256146:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04930a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495940736:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760787.256226:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27422720, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:0.0:1789760787.256238:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547edd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.256262:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760787.256267:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.256275:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547edd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27422720->27426815] (req 27422720->27426815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.256856:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547edd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27422720->27426815] (req 27422720->27426815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.296000:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760787.324140:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974012 is committed 00002000:00010000:1.0:1789760787.324239:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27422720 -> 27426816 00010000:00010000:1.0:1789760787.324259:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547edd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27422720->27426815] (req 27422720->27426815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.324274:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547edd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27422720->27426815] (req 27422720->27426815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.324310:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547edd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27422720->27426815] (req 27422720->27426815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.324319:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760787.324325:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547edd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27422720->27426815] (req 27422720->27426815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.324336:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547edd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27422720->27426815] (req 27422720->27426815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.325581:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547edd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27422720->27426815] (req 27422720->27426815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.325596:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x4936748354547edd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27422720->27426815] (req 27422720->27426815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.325966:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04930a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495940736:12345-192.168.203.9@tcp:4:dd.0 Request processed in 69816us (71299us total) trans 4294974012 rc 0/0 00000100:00100000:0.0:1789760787.325981:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6724 00000100:00100000:0.0:1789760787.325994:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771500935248 time 771576845391 state 0x2 ref 0 00000100:00100000:2.0:1789760787.335022:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760787.335430:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495940864 00000100:00100000:0.0:1789760787.335444:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495940864 00000100:00100000:0.0:1789760787.335528:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771500935248 time 771576845391 state 0x0 ref 1 00000100:00100000:0.0:1789760787.335549:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771577105807 time 771576845391 state 0x1 ref 1 00000100:00100000:0.0:1789760787.335573:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771577105807 time 771656291005 state 0x0 ref 1 00000100:00100000:0.0:1789760787.335587:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04931180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495940864:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760787.335656:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27426816, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:0.0:1789760787.335668:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ee4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.335688:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760787.335694:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.335712:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ee4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27426816->27430911] (req 27426816->27430911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.335728:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ee4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27426816->27430911] (req 27426816->27430911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.365929:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974013 is committed 00002000:00010000:0.0:1789760787.366157:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27426816 -> 27430912 00010000:00010000:0.0:1789760787.366178:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ee4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27426816->27430911] (req 27426816->27430911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.366195:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ee4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27426816->27430911] (req 27426816->27430911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.366205:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ee4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27426816->27430911] (req 27426816->27430911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.366213:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760787.366219:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ee4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27426816->27430911] (req 27426816->27430911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.366230:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ee4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27426816->27430911] (req 27426816->27430911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.366258:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ee4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27426816->27430911] (req 27426816->27430911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.366266:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ee4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27426816->27430911] (req 27426816->27430911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.366490:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495940864:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30900us (31479us total) trans 4294974013 rc 0/0 00000100:00100000:0.0:1789760787.366503:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6725 00000100:00100000:0.0:1789760787.366514:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771577105807 time 771656291005 state 0x2 ref 0 00000100:00100000:3.0:1789760787.378112:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760787.378460:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495940992 00000100:00100000:2.0:1789760787.378472:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495940992 00000100:00100000:2.0:1789760787.378545:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771577105807 time 771656291005 state 0x0 ref 1 00000100:00100000:2.0:1789760787.378565:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771656551421 time 771656291005 state 0x1 ref 1 00000100:00100000:2.0:1789760787.378664:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771656551421 time 771699379345 state 0x0 ref 1 00000100:00100000:2.0:1789760787.378681:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495940992:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760787.378762:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27430912, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:2.0:1789760787.378771:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547eeb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.378792:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:2.0:1789760787.378796:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.378802:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547eeb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27430912->27435007] (req 27430912->27435007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.378814:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547eeb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27430912->27435007] (req 27430912->27435007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.413883:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974014 is committed 00002000:00010000:0.0:1789760787.414092:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27430912 -> 27435008 00010000:00010000:0.0:1789760787.414114:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547eeb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27430912->27435007] (req 27430912->27435007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.414129:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547eeb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27430912->27435007] (req 27430912->27435007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.414139:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547eeb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27430912->27435007] (req 27430912->27435007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.414149:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760787.414154:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547eeb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27430912->27435007] (req 27430912->27435007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.414165:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547eeb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27430912->27435007] (req 27430912->27435007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.414200:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547eeb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27430912->27435007] (req 27430912->27435007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.414209:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354547eeb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27430912->27435007] (req 27430912->27435007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.414380:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495940992:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35695us (36284us total) trans 4294974014 rc 0/0 00000100:00100000:0.0:1789760787.414392:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6726 00000100:00100000:0.0:1789760787.414405:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771656551421 time 771699379345 state 0x2 ref 0 00000100:00100000:2.0:1789760787.418568:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760787.418797:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495941120 00000100:00100000:0.0:1789760787.418810:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495941120 00000100:00100000:0.0:1789760787.418886:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771656551421 time 771699379345 state 0x0 ref 1 00000100:00100000:0.0:1789760787.418902:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771699639761 time 771699379345 state 0x1 ref 1 00000100:00100000:0.0:1789760787.418940:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771699639761 time 771739656220 state 0x0 ref 1 00000100:00100000:0.0:1789760787.418953:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929018a80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495941120:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760787.419019:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27435008, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:0.0:1789760787.419031:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ef2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.419050:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760787.419056:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.419063:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ef2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27435008->27439103] (req 27435008->27439103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.419079:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ef2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27435008->27439103] (req 27435008->27439103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.482509:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974015 is committed 00002000:00010000:0.0:1789760787.482645:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27435008 -> 27439104 00010000:00010000:0.0:1789760787.482669:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ef2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27435008->27439103] (req 27435008->27439103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.482749:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ef2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27435008->27439103] (req 27435008->27439103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.482759:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ef2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27435008->27439103] (req 27435008->27439103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.482767:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760787.482773:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ef2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27435008->27439103] (req 27435008->27439103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.482783:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ef2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27435008->27439103] (req 27435008->27439103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.482808:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ef2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27435008->27439103] (req 27435008->27439103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.482817:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ef2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27435008->27439103] (req 27435008->27439103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.483041:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929018a80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495941120:12345-192.168.203.9@tcp:4:dd.0 Request processed in 64034us (64433us total) trans 4294974015 rc 0/0 00000100:00100000:0.0:1789760787.483058:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6727 00000100:00100000:0.0:1789760787.483070:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771699639761 time 771739656220 state 0x2 ref 0 00000100:00100000:3.0:1789760787.492480:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760787.496040:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495941248 00000100:00100000:0.0:1789760787.496061:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495941248 00000100:00100000:0.0:1789760787.496147:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771699639761 time 771739656220 state 0x0 ref 1 00000100:00100000:0.0:1789760787.496169:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771739916636 time 771739656220 state 0x1 ref 1 00000100:00100000:0.0:1789760787.496213:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771739916636 time 771816928462 state 0x0 ref 1 00000100:00100000:0.0:1789760787.496228:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929005d87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495941248:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760787.496310:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27439104, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:0.0:1789760787.496321:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ef9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.496343:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760787.496349:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.496356:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ef9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27439104->27443199] (req 27439104->27443199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.496373:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ef9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27439104->27443199] (req 27439104->27443199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.530232:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974016 is committed 00002000:00010000:2.0:1789760787.530862:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27439104 -> 27443200 00010000:00010000:2.0:1789760787.530883:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ef9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27439104->27443199] (req 27439104->27443199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.530899:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ef9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27439104->27443199] (req 27439104->27443199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.530908:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ef9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27439104->27443199] (req 27439104->27443199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.530921:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760787.530927:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ef9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27439104->27443199] (req 27439104->27443199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.530937:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ef9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27439104->27443199] (req 27439104->27443199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.530962:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ef9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27439104->27443199] (req 27439104->27443199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.530969:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x4936748354547ef9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27439104->27443199] (req 27439104->27443199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.531089:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929005d87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495941248:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34857us (38615us total) trans 4294974016 rc 0/0 00000100:00100000:2.0:1789760787.531102:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6728 00000100:00100000:2.0:1789760787.531116:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771739916636 time 771816928462 state 0x2 ref 0 00000100:00100000:3.0:1789760787.536591:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760787.537034:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495941376 00000100:00100000:0.0:1789760787.537045:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495941376 00000100:00100000:0.0:1789760787.537107:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771739916636 time 771816928462 state 0x0 ref 1 00000100:00100000:0.0:1789760787.537123:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771817188878 time 771816928462 state 0x1 ref 1 00000100:00100000:0.0:1789760787.537152:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771817188878 time 771857868680 state 0x0 ref 1 00000100:00100000:0.0:1789760787.537168:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495941376:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760787.537217:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27443200, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:0.0:1789760787.537226:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547f00 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.537241:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760787.537245:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.537252:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547f00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27443200->27447295] (req 27443200->27447295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.537266:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547f00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27443200->27447295] (req 27443200->27447295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.617039:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974017 is committed 00002000:00010000:0.0:1789760787.617140:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27443200 -> 27447296 00010000:00010000:0.0:1789760787.617156:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547f00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27443200->27447295] (req 27443200->27447295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.617168:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547f00 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27443200->27447295] (req 27443200->27447295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.617178:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547f00 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27443200->27447295] (req 27443200->27447295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.617186:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760787.617191:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547f00 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27443200->27447295] (req 27443200->27447295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.617200:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547f00 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27443200->27447295] (req 27443200->27447295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.617219:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547f00 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27443200->27447295] (req 27443200->27447295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.617226:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547f00 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27443200->27447295] (req 27443200->27447295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.617446:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929040814000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495941376:12345-192.168.203.9@tcp:4:dd.0 Request processed in 80278us (80859us total) trans 4294974017 rc 0/0 00000100:00100000:0.0:1789760787.617456:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6729 00000100:00100000:0.0:1789760787.617466:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771817188878 time 771857868680 state 0x2 ref 0 00000100:00100000:1.0:1789760787.620889:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760787.620965:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495941504 00000100:00100000:0.0:1789760787.620975:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495941504 00000100:00100000:0.0:1789760787.621029:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771817188878 time 771857868680 state 0x0 ref 1 00000100:00100000:0.0:1789760787.621044:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771858129096 time 771857868680 state 0x1 ref 1 00000100:00100000:0.0:1789760787.621079:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771858129096 time 771941795659 state 0x0 ref 1 00000100:00100000:0.0:1789760787.621096:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495941504:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760787.621148:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27447296, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:0.0:1789760787.621157:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547f07 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.621174:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760787.621178:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.621184:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547f07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27447296->27451391] (req 27447296->27451391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.621197:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547f07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27447296->27451391] (req 27447296->27451391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.645184:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974018 is committed 00002000:00010000:2.0:1789760787.645261:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27447296 -> 27451392 00002000:00010000:2.0:1789760787.645368:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760786 -> 1789760787 00002000:00010000:2.0:1789760787.645370:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760786 -> 1789760787 00010000:00010000:2.0:1789760787.645383:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547f07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27447296->27451391] (req 27447296->27451391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.645396:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547f07 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27447296->27451391] (req 27447296->27451391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.645406:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547f07 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27447296->27451391] (req 27447296->27451391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.645413:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760787.645418:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547f07 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27447296->27451391] (req 27447296->27451391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.645427:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547f07 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27447296->27451391] (req 27447296->27451391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.645445:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547f07 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27447296->27451391] (req 27447296->27451391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.645452:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354547f07 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27447296->27451391] (req 27447296->27451391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.645541:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495941504:12345-192.168.203.9@tcp:4:dd.0 Request processed in 24442us (24657us total) trans 4294974018 rc 0/0 00000100:00100000:2.0:1789760787.645550:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6730 00000100:00100000:2.0:1789760787.645559:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771858129096 time 771941795659 state 0x2 ref 0 00000100:00100000:2.0:1789760787.649474:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760787.649713:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495941632 00000100:00100000:0.0:1789760787.649724:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495941632 00000100:00100000:0.0:1789760787.649781:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771858129096 time 771941795659 state 0x0 ref 1 00000100:00100000:0.0:1789760787.649855:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771942056075 time 771941795659 state 0x1 ref 1 00000100:00100000:0.0:1789760787.649882:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771942056075 time 771970598270 state 0x0 ref 1 00000100:00100000:0.0:1789760787.649894:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495941632:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760787.649942:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27451392, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:0.0:1789760787.649951:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547f0e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.649972:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760787.649977:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.649983:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547f0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27451392->27455487] (req 27451392->27455487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.649997:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547f0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27451392->27455487] (req 27451392->27455487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760787.674067:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27451392 -> 27455488 00010000:00010000:1.0:1789760787.674086:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547f0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27451392->27455487] (req 27451392->27455487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.674100:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547f0e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27451392->27455487] (req 27451392->27455487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.674108:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547f0e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27451392->27455487] (req 27451392->27455487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.674115:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760787.674120:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547f0e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27451392->27455487] (req 27451392->27455487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.674128:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547f0e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27451392->27455487] (req 27451392->27455487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.674147:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547f0e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27451392->27455487] (req 27451392->27455487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.674154:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354547f0e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27451392->27455487] (req 27451392->27455487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.674261:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974019 is committed 00000100:00100000:1.0:1789760787.674323:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495941632:12345-192.168.203.9@tcp:4:dd.0 Request processed in 24381us (24809us total) trans 4294974019 rc 0/0 00000100:00100000:1.0:1789760787.674335:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6731 00000100:00100000:1.0:1789760787.674345:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771942056075 time 771970598270 state 0x2 ref 0 00000100:00100000:3.0:1789760787.680715:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760787.681201:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495941760 00000100:00100000:0.0:1789760787.681212:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495941760 00000100:00100000:0.0:1789760787.681267:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771942056075 time 771970598270 state 0x0 ref 1 00000100:00100000:0.0:1789760787.681324:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771970858686 time 771970598270 state 0x1 ref 1 00000100:00100000:0.0:1789760787.681362:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771970858686 time 772002078236 state 0x0 ref 1 00000100:00100000:0.0:1789760787.681374:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495941760:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760787.681425:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27455488, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:0.0:1789760787.681434:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547f15 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.681450:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:0.0:1789760787.681455:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.681461:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547f15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27455488->27459583] (req 27455488->27459583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.681478:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547f15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27455488->27459583] (req 27455488->27459583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.711100:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974020 is committed 00002000:00010000:1.0:1789760787.711109:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27455488 -> 27459584 00010000:00010000:1.0:1789760787.711125:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547f15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27455488->27459583] (req 27455488->27459583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.711143:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547f15 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27455488->27459583] (req 27455488->27459583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.711153:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547f15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27455488->27459583] (req 27455488->27459583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.711161:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760787.711166:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547f15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27455488->27459583] (req 27455488->27459583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.711175:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547f15 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27455488->27459583] (req 27455488->27459583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.711196:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547f15 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27455488->27459583] (req 27455488->27459583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.711209:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x4936748354547f15 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27455488->27459583] (req 27455488->27459583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.711380:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495941760:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30003us (30680us total) trans 4294974020 rc 0/0 00000100:00100000:1.0:1789760787.711390:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6732 00000100:00100000:1.0:1789760787.711400:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771970858686 time 772002078236 state 0x2 ref 0 00000100:00100000:2.0:1789760787.718058:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760787.718240:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495941888 00000100:00100000:0.0:1789760787.718250:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495941888 00000100:00100000:0.0:1789760787.718337:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 771970858686 time 772002078236 state 0x0 ref 1 00000100:00100000:0.0:1789760787.718353:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772002338652 time 772002078236 state 0x1 ref 1 00000100:00100000:0.0:1789760787.718380:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772002338652 time 772039096569 state 0x0 ref 1 00000100:00100000:0.0:1789760787.718392:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495941888:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760787.718616:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27459584, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:0.0:1789760787.718626:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547f1c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.718643:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:0.0:1789760787.718648:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.718656:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547f1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27459584->27463679] (req 27459584->27463679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.718670:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547f1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27459584->27463679] (req 27459584->27463679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.758374:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974021 is committed 00002000:00010000:1.0:1789760787.758869:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27459584 -> 27463680 00010000:00010000:1.0:1789760787.758894:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547f1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27459584->27463679] (req 27459584->27463679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.758907:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547f1c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27459584->27463679] (req 27459584->27463679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.758915:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547f1c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27459584->27463679] (req 27459584->27463679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.758922:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760787.758927:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547f1c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27459584->27463679] (req 27459584->27463679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.758936:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547f1c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27459584->27463679] (req 27459584->27463679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.758960:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547f1c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27459584->27463679] (req 27459584->27463679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.758968:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x4936748354547f1c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27459584->27463679] (req 27459584->27463679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.759049:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495941888:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40655us (40996us total) trans 4294974021 rc 0/0 00000100:00100000:1.0:1789760787.759059:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6733 00000100:00100000:1.0:1789760787.759069:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772002338652 time 772039096569 state 0x2 ref 0 00000100:00100000:2.0:1789760787.766959:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760787.767080:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495942016 00000100:00100000:1.0:1789760787.767090:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495942016 00000100:00100000:1.0:1789760787.767150:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772002338652 time 772039096569 state 0x0 ref 1 00000100:00100000:1.0:1789760787.767166:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772039356985 time 772039096569 state 0x1 ref 1 00000100:00100000:1.0:1789760787.767230:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772039356985 time 772087946484 state 0x0 ref 1 00000100:00100000:1.0:1789760787.767243:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495942016:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760787.767336:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27463680, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:1.0:1789760787.767345:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547f23 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.767361:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760787.767366:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.767372:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547f23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27463680->27467775] (req 27463680->27467775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.767384:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547f23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27463680->27467775] (req 27463680->27467775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.793495:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974022 is committed 00002000:00010000:2.0:1789760787.793609:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27463680 -> 27467776 00010000:00010000:2.0:1789760787.793624:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547f23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27463680->27467775] (req 27463680->27467775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.793636:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547f23 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27463680->27467775] (req 27463680->27467775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.793645:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547f23 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27463680->27467775] (req 27463680->27467775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.793652:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760787.793658:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547f23 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27463680->27467775] (req 27463680->27467775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.793667:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547f23 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27463680->27467775] (req 27463680->27467775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.793764:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547f23 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27463680->27467775] (req 27463680->27467775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.793772:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547f23 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27463680->27467775] (req 27463680->27467775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.793867:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495942016:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26621us (26914us total) trans 4294974022 rc 0/0 00000100:00100000:2.0:1789760787.793877:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6734 00000100:00100000:2.0:1789760787.793887:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772039356985 time 772087946484 state 0x2 ref 0 00000100:00100000:3.0:1789760787.802252:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760787.802542:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495942144 00000100:00100000:1.0:1789760787.802554:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495942144 00000100:00100000:1.0:1789760787.802606:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772039356985 time 772087946484 state 0x0 ref 1 00000100:00100000:1.0:1789760787.802619:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772088206900 time 772087946484 state 0x1 ref 1 00000100:00100000:1.0:1789760787.802706:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772088206900 time 772123412781 state 0x0 ref 1 00000100:00100000:1.0:1789760787.802728:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495942144:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760787.802776:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27467776, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:1.0:1789760787.802787:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547f2a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.802806:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760787.802811:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.802818:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547f2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27467776->27471871] (req 27467776->27471871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.802834:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547f2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27467776->27471871] (req 27467776->27471871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760787.845332:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27467776 -> 27471872 00010000:00010000:2.0:1789760787.845352:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547f2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27467776->27471871] (req 27467776->27471871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.845364:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547f2a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27467776->27471871] (req 27467776->27471871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.845373:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547f2a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27467776->27471871] (req 27467776->27471871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.845380:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760787.845384:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547f2a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27467776->27471871] (req 27467776->27471871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.845392:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547f2a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27467776->27471871] (req 27467776->27471871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.845412:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547f2a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27467776->27471871] (req 27467776->27471871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.845421:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x4936748354547f2a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27467776->27471871] (req 27467776->27471871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.845594:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495942144:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42873us (43345us total) trans 4294974023 rc 0/0 00000100:00100000:2.0:1789760787.845605:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6735 00000100:00100000:2.0:1789760787.845616:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772088206900 time 772123412781 state 0x2 ref 0 00000001:00080000:3.0:1789760787.846547:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974023 is committed 00000100:00100000:3.0:1789760787.851057:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760787.851184:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495942272 00000100:00100000:1.0:1789760787.851194:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495942272 00000100:00100000:1.0:1789760787.851248:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772088206900 time 772123412781 state 0x0 ref 1 00000100:00100000:1.0:1789760787.851265:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772123673197 time 772123412781 state 0x1 ref 1 00000100:00100000:1.0:1789760787.851338:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772123673197 time 772172049902 state 0x0 ref 1 00000100:00100000:1.0:1789760787.851351:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495942272:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760787.851409:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27471872, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:1.0:1789760787.851419:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547f31 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.851447:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760787.851452:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.851459:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547f31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27471872->27475967] (req 27471872->27475967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.851472:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547f31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27471872->27475967] (req 27471872->27475967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760787.927405:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27471872 -> 27475968 00010000:00010000:1.0:1789760787.927425:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547f31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27471872->27475967] (req 27471872->27475967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.927443:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547f31 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27471872->27475967] (req 27471872->27475967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.927466:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547f31 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27471872->27475967] (req 27471872->27475967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.927474:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760787.927479:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547f31 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27471872->27475967] (req 27471872->27475967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.927488:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547f31 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27471872->27475967] (req 27471872->27475967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.927507:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547f31 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27471872->27475967] (req 27471872->27475967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.927515:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x4936748354547f31 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27471872->27475967] (req 27471872->27475967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.927546:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974024 is committed 00000100:00100000:1.0:1789760787.927596:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495942272:12345-192.168.203.9@tcp:4:dd.0 Request processed in 76243us (76544us total) trans 4294974024 rc 0/0 00000100:00100000:1.0:1789760787.927607:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6736 00000100:00100000:1.0:1789760787.927617:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772123673197 time 772172049902 state 0x2 ref 0 00000100:00100000:1.0:1789760787.935114:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760787.935203:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495942400 00000100:00100000:2.0:1789760787.935215:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495942400 00000100:00100000:2.0:1789760787.935272:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772123673197 time 772172049902 state 0x0 ref 1 00000100:00100000:2.0:1789760787.935332:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772172310318 time 772172049902 state 0x1 ref 1 00000100:00100000:2.0:1789760787.935364:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772172310318 time 772256080529 state 0x0 ref 1 00000100:00100000:2.0:1789760787.935376:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495942400:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760787.935422:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27475968, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:2.0:1789760787.935431:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547f38 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.935447:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:2.0:1789760787.935615:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.935625:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547f38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27475968->27480063] (req 27475968->27480063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.935639:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547f38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27475968->27480063] (req 27475968->27480063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.990005:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974025 is committed 00002000:00010000:1.0:1789760787.991420:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27475968 -> 27480064 00010000:00010000:1.0:1789760787.991441:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547f38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27475968->27480063] (req 27475968->27480063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.991454:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547f38 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27475968->27480063] (req 27475968->27480063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.991462:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547f38 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27475968->27480063] (req 27475968->27480063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.991469:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760787.991473:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547f38 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27475968->27480063] (req 27475968->27480063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.991482:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547f38 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27475968->27480063] (req 27475968->27480063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.991501:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547f38 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27475968->27480063] (req 27475968->27480063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.991509:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354547f38 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27475968->27480063] (req 27475968->27480063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.991681:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495942400:12345-192.168.203.9@tcp:4:dd.0 Request processed in 56292us (56561us total) trans 4294974025 rc 0/0 00000100:00100000:1.0:1789760787.991703:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6737 00000100:00100000:1.0:1789760787.991714:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772172310318 time 772256080529 state 0x2 ref 0 00000100:00100000:3.0:1789760787.996596:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760787.996716:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495942528 00000100:00100000:2.0:1789760787.996725:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495942528 00000100:00100000:2.0:1789760787.996784:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772172310318 time 772256080529 state 0x0 ref 1 00000100:00100000:2.0:1789760787.996798:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772256340945 time 772256080529 state 0x1 ref 1 00000100:00100000:2.0:1789760787.996827:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772256340945 time 772317544605 state 0x0 ref 1 00000100:00100000:2.0:1789760787.996837:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495942528:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760787.996879:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27480064, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:2.0:1789760787.996888:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547f3f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.996902:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:2.0:1789760787.996906:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.996912:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547f3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27480064->27484159] (req 27480064->27484159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.996925:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547f3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27480064->27484159] (req 27480064->27484159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.036414:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974026 is committed 00002000:00010000:3.0:1789760788.036556:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27480064 -> 27484160 00010000:00010000:3.0:1789760788.036571:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547f3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27480064->27484159] (req 27480064->27484159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.036584:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547f3f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27480064->27484159] (req 27480064->27484159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.036593:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547f3f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27480064->27484159] (req 27480064->27484159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.036601:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760788.036605:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547f3f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27480064->27484159] (req 27480064->27484159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.036614:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547f3f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27480064->27484159] (req 27480064->27484159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.036635:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547f3f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27480064->27484159] (req 27480064->27484159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.036643:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354547f3f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27480064->27484159] (req 27480064->27484159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.036749:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495942528:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39907us (40156us total) trans 4294974026 rc 0/0 00000100:00100000:3.0:1789760788.036764:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6738 00000100:00100000:3.0:1789760788.036774:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772256340945 time 772317544605 state 0x2 ref 0 00000100:00100000:2.0:1789760788.043611:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760788.044078:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495942656 00000100:00100000:1.0:1789760788.044089:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495942656 00000100:00100000:1.0:1789760788.044154:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772256340945 time 772317544605 state 0x0 ref 1 00000100:00100000:1.0:1789760788.044172:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772317805021 time 772317544605 state 0x1 ref 1 00000100:00100000:1.0:1789760788.044254:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772317805021 time 772364969852 state 0x0 ref 1 00000100:00100000:1.0:1789760788.044268:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495942656:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760788.044361:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27484160, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:1.0:1789760788.044370:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547f46 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.044388:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760788.044393:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.044399:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547f46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27484160->27488255] (req 27484160->27488255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.044413:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547f46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27484160->27488255] (req 27484160->27488255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.079054:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974027 is committed 00002000:00010000:1.0:1789760788.079132:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27484160 -> 27488256 00010000:00010000:1.0:1789760788.079149:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547f46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27484160->27488255] (req 27484160->27488255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.079168:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547f46 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27484160->27488255] (req 27484160->27488255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.079177:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547f46 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27484160->27488255] (req 27484160->27488255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.079184:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760788.079189:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547f46 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27484160->27488255] (req 27484160->27488255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.079198:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547f46 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27484160->27488255] (req 27484160->27488255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.079219:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547f46 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27484160->27488255] (req 27484160->27488255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.079226:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547f46 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27484160->27488255] (req 27484160->27488255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.079370:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495942656:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35099us (35765us total) trans 4294974027 rc 0/0 00000100:00100000:1.0:1789760788.079381:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6739 00000100:00100000:1.0:1789760788.079392:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772317805021 time 772364969852 state 0x2 ref 0 00000100:00100000:1.0:1789760788.090161:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760788.090308:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495942784 00000100:00100000:2.0:1789760788.090318:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495942784 00000100:00100000:2.0:1789760788.090370:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772317805021 time 772364969852 state 0x0 ref 1 00000100:00100000:2.0:1789760788.090385:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772365230268 time 772364969852 state 0x1 ref 1 00000100:00100000:2.0:1789760788.090407:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772365230268 time 772411124042 state 0x0 ref 1 00000100:00100000:2.0:1789760788.090419:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495942784:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760788.090468:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27488256, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:2.0:1789760788.090476:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547f4d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.090491:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:2.0:1789760788.090496:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.090503:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547f4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27488256->27492351] (req 27488256->27492351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.090518:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547f4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27488256->27492351] (req 27488256->27492351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760788.149051:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27488256 -> 27492352 00010000:00010000:2.0:1789760788.149077:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547f4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27488256->27492351] (req 27488256->27492351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.149089:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547f4d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27488256->27492351] (req 27488256->27492351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.149099:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547f4d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27488256->27492351] (req 27488256->27492351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.149107:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760788.149112:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547f4d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27488256->27492351] (req 27488256->27492351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.149122:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547f4d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27488256->27492351] (req 27488256->27492351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.149311:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547f4d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27488256->27492351] (req 27488256->27492351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.149322:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354547f4d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27488256->27492351] (req 27488256->27492351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.149420:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495942784:12345-192.168.203.9@tcp:4:dd.0 Request processed in 58997us (59262us total) trans 4294974028 rc 0/0 00000100:00100000:2.0:1789760788.149430:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6740 00000100:00100000:2.0:1789760788.149441:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772365230268 time 772411124042 state 0x2 ref 0 00000001:00080000:3.0:1789760788.151084:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974028 is committed 00000100:00100000:1.0:1789760788.155656:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760788.157203:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495942912 00000100:00100000:3.0:1789760788.157216:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495942912 00000100:00100000:3.0:1789760788.157277:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772365230268 time 772411124042 state 0x0 ref 1 00000100:00100000:3.0:1789760788.157308:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772411384458 time 772411124042 state 0x1 ref 1 00000100:00100000:3.0:1789760788.157342:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772411384458 time 772478058656 state 0x0 ref 1 00000100:00100000:3.0:1789760788.157354:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495942912:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760788.157402:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27492352, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:3.0:1789760788.157411:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547f54 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.157426:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760788.157430:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.157437:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547f54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27492352->27496447] (req 27492352->27496447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.157600:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547f54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27492352->27496447] (req 27492352->27496447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.190742:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974029 is committed 00002000:00010000:3.0:1789760788.190801:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27492352 -> 27496448 00010000:00010000:3.0:1789760788.190815:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547f54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27492352->27496447] (req 27492352->27496447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.190828:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547f54 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27492352->27496447] (req 27492352->27496447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.190835:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547f54 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27492352->27496447] (req 27492352->27496447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.190842:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760788.190846:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547f54 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27492352->27496447] (req 27492352->27496447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.190854:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547f54 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27492352->27496447] (req 27492352->27496447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.190872:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547f54 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27492352->27496447] (req 27492352->27496447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.190879:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547f54 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27492352->27496447] (req 27492352->27496447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.190976:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495942912:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33618us (35325us total) trans 4294974029 rc 0/0 00000100:00100000:3.0:1789760788.190987:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6741 00000100:00100000:3.0:1789760788.190999:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772411384458 time 772478058656 state 0x2 ref 0 00000100:00100000:0.0:1789760788.195063:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760788.195427:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495943040 00000100:00100000:1.0:1789760788.195440:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495943040 00000100:00100000:1.0:1789760788.195827:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772411384458 time 772478058656 state 0x0 ref 1 00000100:00100000:1.0:1789760788.195844:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772478319072 time 772478058656 state 0x1 ref 1 00000100:00100000:1.0:1789760788.195913:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772478319072 time 772516629202 state 0x0 ref 1 00000100:00100000:1.0:1789760788.195926:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01377800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495943040:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760788.195982:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27496448, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:1.0:1789760788.195992:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547f5b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.196007:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760788.196012:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.196019:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547f5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27496448->27500543] (req 27496448->27500543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.196033:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547f5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27496448->27500543] (req 27496448->27500543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.243386:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974030 is committed 00002000:00010000:0.0:1789760788.243555:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27496448 -> 27500544 00010000:00010000:0.0:1789760788.243572:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547f5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27496448->27500543] (req 27496448->27500543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.243586:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547f5b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27496448->27500543] (req 27496448->27500543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.243595:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547f5b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27496448->27500543] (req 27496448->27500543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.243602:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760788.243606:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547f5b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27496448->27500543] (req 27496448->27500543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.243615:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547f5b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27496448->27500543] (req 27496448->27500543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.243636:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547f5b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27496448->27500543] (req 27496448->27500543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.243644:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x4936748354547f5b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27496448->27500543] (req 27496448->27500543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.243865:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01377800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495943040:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47936us (49426us total) trans 4294974030 rc 0/0 00000100:00100000:0.0:1789760788.243876:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6742 00000100:00100000:0.0:1789760788.243886:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772478319072 time 772516629202 state 0x2 ref 0 00000100:00100000:2.0:1789760788.247419:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760788.247569:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495943168 00000100:00100000:1.0:1789760788.247579:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495943168 00000100:00100000:1.0:1789760788.247634:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772478319072 time 772516629202 state 0x0 ref 1 00000100:00100000:1.0:1789760788.247650:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772516889618 time 772516629202 state 0x1 ref 1 00000100:00100000:1.0:1789760788.247687:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772516889618 time 772568403727 state 0x0 ref 1 00000100:00100000:1.0:1789760788.247711:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495943168:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760788.247761:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27500544, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:1.0:1789760788.247769:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547f62 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.247786:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760788.247790:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.247796:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547f62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27500544->27504639] (req 27500544->27504639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.247809:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547f62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27500544->27504639] (req 27500544->27504639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.275376:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974031 is committed 00002000:00010000:0.0:1789760788.275563:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27500544 -> 27504640 00010000:00010000:0.0:1789760788.275583:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547f62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27500544->27504639] (req 27500544->27504639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.275595:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547f62 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27500544->27504639] (req 27500544->27504639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.275605:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547f62 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27500544->27504639] (req 27500544->27504639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.275613:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760788.275618:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547f62 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27500544->27504639] (req 27500544->27504639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.275627:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547f62 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27500544->27504639] (req 27500544->27504639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.275650:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547f62 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27500544->27504639] (req 27500544->27504639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.275659:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354547f62 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27500544->27504639] (req 27500544->27504639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.275869:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495943168:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28154us (28454us total) trans 4294974031 rc 0/0 00000100:00100000:0.0:1789760788.275880:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6743 00000100:00100000:0.0:1789760788.275890:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772516889618 time 772568403727 state 0x2 ref 0 00000100:00100000:1.0:1789760788.279920:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760788.280076:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495943296 00000100:00100000:3.0:1789760788.280086:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495943296 00000100:00100000:3.0:1789760788.280140:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772516889618 time 772568403727 state 0x0 ref 1 00000100:00100000:3.0:1789760788.280155:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772568664143 time 772568403727 state 0x1 ref 1 00000100:00100000:3.0:1789760788.280221:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772568664143 time 772600937570 state 0x0 ref 1 00000100:00100000:3.0:1789760788.280233:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495943296:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760788.280299:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27504640, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:3.0:1789760788.280308:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547f69 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.280323:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760788.280328:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.280334:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547f69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27504640->27508735] (req 27504640->27508735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.280346:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547f69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27504640->27508735] (req 27504640->27508735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760788.313024:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27504640 -> 27508736 00010000:00010000:0.0:1789760788.313044:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547f69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27504640->27508735] (req 27504640->27508735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.313056:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547f69 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27504640->27508735] (req 27504640->27508735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.313065:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547f69 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27504640->27508735] (req 27504640->27508735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.313073:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760788.313077:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547f69 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27504640->27508735] (req 27504640->27508735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.313086:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547f69 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27504640->27508735] (req 27504640->27508735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.313107:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547f69 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27504640->27508735] (req 27504640->27508735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.313121:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354547f69 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27504640->27508735] (req 27504640->27508735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.313225:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495943296:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32988us (33309us total) trans 4294974032 rc 0/0 00000100:00100000:0.0:1789760788.313236:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6744 00000100:00100000:0.0:1789760788.313247:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772568664143 time 772600937570 state 0x2 ref 0 00000001:00080000:2.0:1789760788.316047:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974032 is committed 00000100:00100000:2.0:1789760788.316609:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760788.316761:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495943424 00000100:00100000:0.0:1789760788.316770:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495943424 00000100:00100000:0.0:1789760788.316823:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772568664143 time 772600937570 state 0x0 ref 1 00000100:00100000:0.0:1789760788.316838:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772601197986 time 772600937570 state 0x1 ref 1 00000100:00100000:0.0:1789760788.316908:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772601197986 time 772637624024 state 0x0 ref 1 00000100:00100000:0.0:1789760788.316920:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495943424:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760788.316967:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27508736, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:0.0:1789760788.316975:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f70 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.316991:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:0.0:1789760788.316995:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.317001:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27508736->27512831] (req 27508736->27512831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.317027:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27508736->27512831] (req 27508736->27512831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.323500:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760788.347726:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27508736 -> 27512832 00010000:00010000:0.0:1789760788.347744:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27508736->27512831] (req 27508736->27512831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.347755:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f70 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27508736->27512831] (req 27508736->27512831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.347763:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f70 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27508736->27512831] (req 27508736->27512831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.347770:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760788.347774:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f70 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27508736->27512831] (req 27508736->27512831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.347782:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f70 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27508736->27512831] (req 27508736->27512831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.347801:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f70 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27508736->27512831] (req 27508736->27512831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.347809:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f70 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27508736->27512831] (req 27508736->27512831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.347906:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495943424:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30983us (31297us total) trans 4294974033 rc 0/0 00000100:00100000:0.0:1789760788.347917:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6745 00000100:00100000:0.0:1789760788.347926:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772601197986 time 772637624024 state 0x2 ref 0 00000001:00080000:2.0:1789760788.348002:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974033 is committed 00000100:00100000:1.0:1789760788.352916:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760788.353470:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495943552 00000100:00100000:0.0:1789760788.353480:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495943552 00000100:00100000:0.0:1789760788.353536:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772601197986 time 772637624024 state 0x0 ref 1 00000100:00100000:0.0:1789760788.353552:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772637884440 time 772637624024 state 0x1 ref 1 00000100:00100000:0.0:1789760788.353583:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772637884440 time 772674299709 state 0x0 ref 1 00000100:00100000:0.0:1789760788.353596:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495943552:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760788.353644:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27512832, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:0.0:1789760788.353652:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547f77 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.353786:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:0.0:1789760788.353792:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.353799:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547f77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27512832->27516927] (req 27512832->27516927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.353812:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547f77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27512832->27516927] (req 27512832->27516927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.389051:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974034 is committed 00002000:00010000:0.0:1789760788.389124:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27512832 -> 27516928 00010000:00010000:0.0:1789760788.389140:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547f77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27512832->27516927] (req 27512832->27516927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.389153:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547f77 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27512832->27516927] (req 27512832->27516927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.389162:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547f77 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27512832->27516927] (req 27512832->27516927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.389170:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760788.389175:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547f77 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27512832->27516927] (req 27512832->27516927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.389184:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547f77 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27512832->27516927] (req 27512832->27516927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.389204:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547f77 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27512832->27516927] (req 27512832->27516927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.389213:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547f77 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27512832->27516927] (req 27512832->27516927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.389372:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033cea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495943552:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35772us (36463us total) trans 4294974034 rc 0/0 00000100:00100000:0.0:1789760788.389386:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6746 00000100:00100000:0.0:1789760788.389396:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772637884440 time 772674299709 state 0x2 ref 0 00000100:00100000:1.0:1789760788.393486:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760788.393983:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495943680 00000100:00100000:0.0:1789760788.393997:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495943680 00000100:00100000:0.0:1789760788.394107:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772637884440 time 772674299709 state 0x0 ref 1 00000100:00100000:0.0:1789760788.394123:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772674560125 time 772674299709 state 0x1 ref 1 00000100:00100000:0.0:1789760788.394782:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772674560125 time 772715497171 state 0x0 ref 1 00000100:00100000:0.0:1789760788.394798:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495943680:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760788.394848:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27516928, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:0.0:1789760788.394856:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f7e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.394871:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:0.0:1789760788.394875:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.394881:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27516928->27521023] (req 27516928->27521023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.394893:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27516928->27521023] (req 27516928->27521023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760788.422093:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27516928 -> 27521024 00010000:00010000:0.0:1789760788.422113:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27516928->27521023] (req 27516928->27521023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.422135:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f7e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27516928->27521023] (req 27516928->27521023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.422143:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f7e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27516928->27521023] (req 27516928->27521023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.422150:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760788.422155:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f7e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27516928->27521023] (req 27516928->27521023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.422164:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f7e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27516928->27521023] (req 27516928->27521023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.422184:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f7e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27516928->27521023] (req 27516928->27521023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.422192:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f7e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27516928->27521023] (req 27516928->27521023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.422322:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495943680:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27486us (28804us total) trans 4294974035 rc 0/0 00000100:00100000:0.0:1789760788.422333:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6747 00000100:00100000:0.0:1789760788.422352:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772674560125 time 772715497171 state 0x2 ref 0 00000001:00080000:1.0:1789760788.423022:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974035 is committed 00000100:00100000:2.0:1789760788.429980:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760788.430381:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495943808 00000100:00100000:0.0:1789760788.430392:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495943808 00000100:00100000:0.0:1789760788.430555:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772674560125 time 772715497171 state 0x0 ref 1 00000100:00100000:0.0:1789760788.430571:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772715757587 time 772715497171 state 0x1 ref 1 00000100:00100000:0.0:1789760788.430606:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772715757587 time 772751321913 state 0x0 ref 1 00000100:00100000:0.0:1789760788.430619:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495943808:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760788.430667:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27521024, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:0.0:1789760788.430676:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f85 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.430706:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:0.0:1789760788.430712:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.430719:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27521024->27525119] (req 27521024->27525119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.430732:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27521024->27525119] (req 27521024->27525119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760788.469531:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27521024 -> 27525120 00010000:00010000:0.0:1789760788.469549:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27521024->27525119] (req 27521024->27525119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.469560:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f85 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27521024->27525119] (req 27521024->27525119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.469573:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f85 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27521024->27525119] (req 27521024->27525119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.469581:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760788.469585:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f85 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27521024->27525119] (req 27521024->27525119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.469595:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f85 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27521024->27525119] (req 27521024->27525119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.469614:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f85 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27521024->27525119] (req 27521024->27525119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.469622:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f85 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27521024->27525119] (req 27521024->27525119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.469888:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974036 is committed 00000100:00100000:0.0:1789760788.469943:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495943808:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39322us (39965us total) trans 4294974036 rc 0/0 00000100:00100000:0.0:1789760788.469952:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6748 00000100:00100000:0.0:1789760788.469961:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772715757587 time 772751321913 state 0x2 ref 0 00000100:00100000:2.0:1789760788.479827:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760788.479910:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495943936 00000100:00100000:0.0:1789760788.479920:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495943936 00000100:00100000:0.0:1789760788.479972:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772715757587 time 772751321913 state 0x0 ref 1 00000100:00100000:0.0:1789760788.479986:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772751582329 time 772751321913 state 0x1 ref 1 00000100:00100000:0.0:1789760788.480017:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772751582329 time 772800733626 state 0x0 ref 1 00000100:00100000:0.0:1789760788.480033:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495943936:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760788.480085:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27525120, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:0.0:1789760788.480095:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f8c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.480111:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:0.0:1789760788.480116:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.480122:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27525120->27529215] (req 27525120->27529215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.480136:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27525120->27529215] (req 27525120->27529215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.507793:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974037 is committed 00002000:00010000:0.0:1789760788.507990:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27525120 -> 27529216 00010000:00010000:0.0:1789760788.508005:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27525120->27529215] (req 27525120->27529215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.508019:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f8c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27525120->27529215] (req 27525120->27529215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.508027:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f8c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27525120->27529215] (req 27525120->27529215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.508034:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760788.508040:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f8c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27525120->27529215] (req 27525120->27529215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.508049:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f8c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27525120->27529215] (req 27525120->27529215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.508070:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f8c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27525120->27529215] (req 27525120->27529215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.508078:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f8c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27525120->27529215] (req 27525120->27529215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.508252:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495943936:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28218us (28439us total) trans 4294974037 rc 0/0 00000100:00100000:0.0:1789760788.508263:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6749 00000100:00100000:0.0:1789760788.508273:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772751582329 time 772800733626 state 0x2 ref 0 00000100:00100000:1.0:1789760788.511532:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760788.511634:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495944064 00000100:00100000:0.0:1789760788.511890:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495944064 00000100:00100000:0.0:1789760788.511958:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772751582329 time 772800733626 state 0x0 ref 1 00000100:00100000:0.0:1789760788.511974:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772800994042 time 772800733626 state 0x1 ref 1 00000100:00100000:0.0:1789760788.512006:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772800994042 time 772832723315 state 0x0 ref 1 00000100:00100000:0.0:1789760788.512019:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495944064:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760788.512066:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27529216, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:0.0:1789760788.512076:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547f93 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.512093:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:0.0:1789760788.512098:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.512105:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547f93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27529216->27533311] (req 27529216->27533311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.512123:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547f93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27529216->27533311] (req 27529216->27533311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.556940:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974038 is committed 00002000:00010000:3.0:1789760788.557044:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27529216 -> 27533312 00010000:00010000:3.0:1789760788.557060:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547f93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27529216->27533311] (req 27529216->27533311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.557074:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547f93 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27529216->27533311] (req 27529216->27533311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.557084:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547f93 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27529216->27533311] (req 27529216->27533311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.557091:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760788.557096:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547f93 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27529216->27533311] (req 27529216->27533311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.557104:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547f93 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27529216->27533311] (req 27529216->27533311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.557124:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547f93 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27529216->27533311] (req 27529216->27533311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.557132:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354547f93 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27529216->27533311] (req 27529216->27533311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.557224:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495944064:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45202us (45698us total) trans 4294974038 rc 0/0 00000100:00100000:3.0:1789760788.557235:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6750 00000100:00100000:3.0:1789760788.557245:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772800994042 time 772832723315 state 0x2 ref 0 00000100:00100000:2.0:1789760788.563055:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760788.563264:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495944192 00000100:00100000:0.0:1789760788.563273:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495944192 00000100:00100000:0.0:1789760788.563360:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772800994042 time 772832723315 state 0x0 ref 1 00000100:00100000:0.0:1789760788.563375:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772832983731 time 772832723315 state 0x1 ref 1 00000100:00100000:0.0:1789760788.563405:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772832983731 time 772884122365 state 0x0 ref 1 00000100:00100000:0.0:1789760788.563417:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fba3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495944192:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760788.563462:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27533312, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:0.0:1789760788.563471:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f9a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.563486:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:0.0:1789760788.563491:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.563497:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27533312->27537407] (req 27533312->27537407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.563511:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27533312->27537407] (req 27533312->27537407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.606136:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974039 is committed 00002000:00010000:0.0:1789760788.606243:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27533312 -> 27537408 00010000:00010000:0.0:1789760788.606267:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27533312->27537407] (req 27533312->27537407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.606308:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f9a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27533312->27537407] (req 27533312->27537407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.606318:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f9a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27533312->27537407] (req 27533312->27537407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.606324:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760788.606330:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f9a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27533312->27537407] (req 27533312->27537407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.606338:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f9a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27533312->27537407] (req 27533312->27537407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.606358:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f9a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27533312->27537407] (req 27533312->27537407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.606366:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547f9a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27533312->27537407] (req 27533312->27537407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.606451:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fba3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495944192:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43031us (43401us total) trans 4294974039 rc 0/0 00000100:00100000:0.0:1789760788.606461:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6751 00000100:00100000:0.0:1789760788.606471:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772832983731 time 772884122365 state 0x2 ref 0 00000100:00100000:2.0:1789760788.611277:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760788.611418:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495944320 00000100:00100000:0.0:1789760788.611429:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495944320 00000100:00100000:0.0:1789760788.611488:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772832983731 time 772884122365 state 0x0 ref 1 00000100:00100000:0.0:1789760788.611502:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772884382781 time 772884122365 state 0x1 ref 1 00000100:00100000:0.0:1789760788.611536:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772884382781 time 772932252585 state 0x0 ref 1 00000100:00100000:0.0:1789760788.611548:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495944320:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760788.611592:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27537408, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:0.0:1789760788.611602:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fa1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.611617:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:0.0:1789760788.611621:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.611627:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fa1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27537408->27541503] (req 27537408->27541503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.611642:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fa1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27537408->27541503] (req 27537408->27541503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.647194:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974040 is committed 00002000:00010000:0.0:1789760788.647345:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27537408 -> 27541504 00010000:00010000:0.0:1789760788.647360:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fa1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27537408->27541503] (req 27537408->27541503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.647373:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fa1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27537408->27541503] (req 27537408->27541503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.647382:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fa1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27537408->27541503] (req 27537408->27541503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.647389:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760788.647394:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fa1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27537408->27541503] (req 27537408->27541503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.647403:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fa1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27537408->27541503] (req 27537408->27541503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.647421:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fa1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27537408->27541503] (req 27537408->27541503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.647428:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fa1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27537408->27541503] (req 27537408->27541503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.647510:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495944320:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35959us (36237us total) trans 4294974040 rc 0/0 00000100:00100000:0.0:1789760788.647524:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6752 00000100:00100000:0.0:1789760788.647534:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772884382781 time 772932252585 state 0x2 ref 0 00000100:00100000:1.0:1789760788.654755:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760788.654999:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495944448 00000100:00100000:3.0:1789760788.655009:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495944448 00000100:00100000:3.0:1789760788.655076:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772884382781 time 772932252585 state 0x0 ref 1 00000100:00100000:3.0:1789760788.655094:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772932513001 time 772932252585 state 0x1 ref 1 00000100:00100000:3.0:1789760788.655122:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772932513001 time 772975839044 state 0x0 ref 1 00000100:00100000:3.0:1789760788.655135:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495944448:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760788.655179:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27541504, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:3.0:1789760788.655187:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547fa8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.655204:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760788.655208:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.655215:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547fa8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27541504->27545599] (req 27541504->27545599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.655228:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547fa8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27541504->27545599] (req 27541504->27545599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.711205:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974041 is committed 00002000:00010000:0.0:1789760788.711420:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27541504 -> 27545600 00002000:00010000:0.0:1789760788.711426:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760787 -> 1789760788 00002000:00010000:0.0:1789760788.711429:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760787 -> 1789760788 00010000:00010000:0.0:1789760788.711443:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547fa8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27541504->27545599] (req 27541504->27545599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.711456:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547fa8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27541504->27545599] (req 27541504->27545599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.711465:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547fa8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27541504->27545599] (req 27541504->27545599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.711473:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760788.711478:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547fa8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27541504->27545599] (req 27541504->27545599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.711489:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547fa8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27541504->27545599] (req 27541504->27545599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.711511:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547fa8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27541504->27545599] (req 27541504->27545599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.711520:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354547fa8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27541504->27545599] (req 27541504->27545599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.711605:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495944448:12345-192.168.203.9@tcp:4:dd.0 Request processed in 56468us (56856us total) trans 4294974041 rc 0/0 00000100:00100000:0.0:1789760788.711615:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6753 00000100:00100000:0.0:1789760788.711626:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772932513001 time 772975839044 state 0x2 ref 0 00000100:00100000:3.0:1789760788.716660:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760788.716759:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495944576 00000100:00100000:0.0:1789760788.716769:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495944576 00000100:00100000:0.0:1789760788.716832:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772932513001 time 772975839044 state 0x0 ref 1 00000100:00100000:0.0:1789760788.716847:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772976099460 time 772975839044 state 0x1 ref 1 00000100:00100000:0.0:1789760788.716877:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772976099460 time 773037593982 state 0x0 ref 1 00000100:00100000:0.0:1789760788.716889:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495944576:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760788.716936:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27545600, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:0.0:1789760788.716944:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547faf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.716960:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760788.716965:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.716971:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547faf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27545600->27549695] (req 27545600->27549695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.716985:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547faf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27545600->27549695] (req 27545600->27549695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.747457:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974042 is committed 00002000:00010000:0.0:1789760788.747554:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27545600 -> 27549696 00010000:00010000:0.0:1789760788.747570:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547faf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27545600->27549695] (req 27545600->27549695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.747583:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547faf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27545600->27549695] (req 27545600->27549695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.747593:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547faf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27545600->27549695] (req 27545600->27549695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.747599:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760788.747603:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547faf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27545600->27549695] (req 27545600->27549695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.747611:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547faf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27545600->27549695] (req 27545600->27549695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.747632:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547faf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27545600->27549695] (req 27545600->27549695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.747638:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547faf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27545600->27549695] (req 27545600->27549695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.747847:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495944576:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30955us (31202us total) trans 4294974042 rc 0/0 00000100:00100000:0.0:1789760788.747862:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6754 00000100:00100000:0.0:1789760788.747873:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772976099460 time 773037593982 state 0x2 ref 0 00000100:00100000:2.0:1789760788.755760:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760788.759615:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495944704 00000100:00100000:0.0:1789760788.759630:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495944704 00000100:00100000:0.0:1789760788.759703:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 772976099460 time 773037593982 state 0x0 ref 1 00000100:00100000:0.0:1789760788.759721:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773037854398 time 773037593982 state 0x1 ref 1 00000100:00100000:0.0:1789760788.759746:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773037854398 time 773080463408 state 0x0 ref 1 00000100:00100000:0.0:1789760788.759759:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495944704:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760788.759807:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27549696, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:0.0:1789760788.759944:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fb6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.759963:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760788.759968:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.759976:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27549696->27553791] (req 27549696->27553791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.759991:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27549696->27553791] (req 27549696->27553791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.766589:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f043bea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:59c4e264-9b75-482b-8813-3d1dc124f384:3317:1876699520316288:0@lo:400:kworker.0 00000100:00100000:1.0:1789760788.766659:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760788.766727:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f03593b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3317:1876699520316800:0@lo:400:kworker.0 00000100:00100000:1.0:1789760788.766753:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760788.766779:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1789760788.766785:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f043bf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3317:1876699520316416:0@lo:400:kworker.0 00000100:00100000:1.0:1789760788.766807:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760788.767022:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff929036e57100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3317:1876699520316928:0@lo:400:kworker.0 00000100:00100000:1.0:1789760788.767049:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760788.767078:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f03592680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520316544:0@lo:400:kworker.0 00000100:00100000:1.0:1789760788.767100:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760788.767116:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760788.767121:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f03590380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520316672:0@lo:400:kworker.0 00000100:00100000:1.0:1789760788.767138:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760788.767176:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520316288 00000100:00100000:1.0:1789760788.767183:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520316288 00000100:00100000:1.0:1789760788.767224:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 419 00000100:00100000:1.0:1789760788.767232:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290122cc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:59c4e264-9b75-482b-8813-3d1dc124f384+19:3317:x1876699520316288:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760788.767312:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290122cc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:59c4e264-9b75-482b-8813-3d1dc124f384+19:3317:x1876699520316288:12345-0@lo:400:kworker.0 Request processed in 79us (651us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760788.767320:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 419 00000100:00100000:1.0:1789760788.767351:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520316800 00000100:00100000:1.0:1789760788.767354:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520316800 00000100:00100000:1.0:1789760788.767384:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520316416 00000100:00100000:1.0:1789760788.767387:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520316416 00000100:00100000:1.0:1789760788.767403:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520316544 00000100:00100000:1.0:1789760788.767406:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520316544 00000100:00100000:1.0:1789760788.767420:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 527 00000100:00100000:1.0:1789760788.767426:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036e55500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1876699520316800:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760788.767459:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036e55500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1876699520316800:12345-0@lo:400:kworker.0 Request processed in 33us (704us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760788.767465:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 527 00000100:00100000:1.0:1789760788.767477:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 528 00000100:00100000:1.0:1789760788.767484:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1876699520316416:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760788.767508:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01836680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1876699520316416:12345-0@lo:400:kworker.0 Request processed in 26us (702us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760788.767514:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 528 00000100:00100000:1.0:1789760788.767524:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 529 00000100:00100000:1.0:1789760788.767530:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520316544:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760788.767556:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3319:x1876699520316544:12345-0@lo:400:kworker.0 Request processed in 26us (455us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760788.767561:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 529 00000100:00100000:1.0:1789760788.767584:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520316928 00000100:00100000:1.0:1789760788.767587:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520316928 00000100:00100000:1.0:1789760788.767611:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520316672 00000100:00100000:1.0:1789760788.767614:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520316672 00000100:00100000:1.0:1789760788.767631:0:6709:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 242 00000100:00100000:1.0:1789760788.767637:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3317:x1876699520316928:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760788.767671:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3317:x1876699520316928:12345-0@lo:400:kworker.0 Request processed in 28us (616us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760788.767677:0:6709:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 242 00000100:00100000:0.0:1789760788.768118:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f043bf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3317:1876699520316416:0@lo:400:kworker.0 00000100:00100000:0.0:1789760788.768130:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff929036e57100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3317:1876699520316928:0@lo:400:kworker.0 00000100:00100000:0.0:1789760788.768138:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f043bea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:59c4e264-9b75-482b-8813-3d1dc124f384:3317:1876699520316288:0@lo:400:kworker.0 00000100:00100000:1.0:1789760788.768542:0:6709:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 243 00000100:00100000:1.0:1789760788.768554:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520316672:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760788.768805:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520316672:12345-0@lo:400:kworker.0 Request processed in 253us (1663us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760788.768881:0:6709:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 243 00000100:00100000:1.0:1789760788.768927:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f03590380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520316672:0@lo:400:kworker.0 00000100:00100000:1.0:1789760788.768938:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f03592680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3319:1876699520316544:0@lo:400:kworker.0 00000100:00100000:0.0:1789760788.769045:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f03593b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3317:1876699520316800:0@lo:400:kworker.0 00000100:00080000:2.0:1789760788.769709:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789760788.769715:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760788.769718:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:2.0:1789760788.808676:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974043 is committed 00002000:00010000:0.0:1789760788.809004:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27549696 -> 27553792 00010000:00010000:0.0:1789760788.809020:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27549696->27553791] (req 27549696->27553791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.809032:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fb6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27549696->27553791] (req 27549696->27553791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.809042:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27549696->27553791] (req 27549696->27553791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.809049:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760788.809055:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27549696->27553791] (req 27549696->27553791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.809064:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fb6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27549696->27553791] (req 27549696->27553791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.809083:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fb6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27549696->27553791] (req 27549696->27553791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.809092:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fb6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27549696->27553791] (req 27549696->27553791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.809180:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495944704:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49419us (53426us total) trans 4294974043 rc 0/0 00000100:00100000:0.0:1789760788.809191:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6755 00000100:00100000:0.0:1789760788.809200:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773037854398 time 773080463408 state 0x2 ref 0 00000100:00100000:1.0:1789760788.814021:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760788.814135:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495944832 00000100:00100000:0.0:1789760788.814145:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495944832 00000100:00100000:0.0:1789760788.814203:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773037854398 time 773080463408 state 0x0 ref 1 00000100:00100000:0.0:1789760788.814223:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773080723824 time 773080463408 state 0x1 ref 1 00000100:00100000:0.0:1789760788.814322:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773080723824 time 773135038111 state 0x0 ref 1 00000100:00100000:0.0:1789760788.814334:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495944832:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760788.814376:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27553792, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:0.0:1789760788.814385:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fbd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.814405:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760788.814410:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.814416:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fbd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27553792->27557887] (req 27553792->27557887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.814433:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fbd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27553792->27557887] (req 27553792->27557887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.852346:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974044 is committed 00002000:00010000:0.0:1789760788.852449:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27553792 -> 27557888 00010000:00010000:0.0:1789760788.852465:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fbd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27553792->27557887] (req 27553792->27557887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.852478:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fbd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27553792->27557887] (req 27553792->27557887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.852487:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fbd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27553792->27557887] (req 27553792->27557887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.852494:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760788.852499:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fbd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27553792->27557887] (req 27553792->27557887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.852508:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fbd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27553792->27557887] (req 27553792->27557887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.852529:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fbd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27553792->27557887] (req 27553792->27557887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.852537:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fbd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27553792->27557887] (req 27553792->27557887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.852631:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495944832:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38293us (38614us total) trans 4294974044 rc 0/0 00000100:00100000:0.0:1789760788.852649:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6756 00000100:00100000:0.0:1789760788.852659:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773080723824 time 773135038111 state 0x2 ref 0 00000100:00100000:2.0:1789760788.856718:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760788.857061:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495944960 00000100:00100000:0.0:1789760788.857073:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495944960 00000100:00100000:0.0:1789760788.857128:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773080723824 time 773135038111 state 0x0 ref 1 00000100:00100000:0.0:1789760788.857152:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773135298527 time 773135038111 state 0x1 ref 1 00000100:00100000:0.0:1789760788.857186:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773135298527 time 773177902904 state 0x0 ref 1 00000100:00100000:0.0:1789760788.857198:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495944960:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760788.857242:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27557888, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:0.0:1789760788.857251:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fc4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.857266:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760788.857271:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.857309:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fc4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27557888->27561983] (req 27557888->27561983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.857324:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fc4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27557888->27561983] (req 27557888->27561983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.897337:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974045 is committed 00002000:00010000:0.0:1789760788.897506:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27557888 -> 27561984 00010000:00010000:0.0:1789760788.897523:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fc4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27557888->27561983] (req 27557888->27561983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.897536:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fc4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27557888->27561983] (req 27557888->27561983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.897550:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fc4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27557888->27561983] (req 27557888->27561983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.897558:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760788.897563:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fc4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27557888->27561983] (req 27557888->27561983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.897571:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fc4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27557888->27561983] (req 27557888->27561983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.897594:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fc4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27557888->27561983] (req 27557888->27561983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.897602:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fc4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27557888->27561983] (req 27557888->27561983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.897718:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495944960:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40518us (41006us total) trans 4294974045 rc 0/0 00000100:00100000:0.0:1789760788.897729:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6757 00000100:00100000:0.0:1789760788.897739:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773135298527 time 773177902904 state 0x2 ref 0 00000100:00100000:2.0:1789760788.906812:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760788.907061:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495945088 00000100:00100000:0.0:1789760788.907071:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495945088 00000100:00100000:0.0:1789760788.907125:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773135298527 time 773177902904 state 0x0 ref 1 00000100:00100000:0.0:1789760788.907140:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773178163320 time 773177902904 state 0x1 ref 1 00000100:00100000:0.0:1789760788.907173:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773178163320 time 773227889974 state 0x0 ref 1 00000100:00100000:0.0:1789760788.907186:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495945088:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760788.907234:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27561984, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:0.0:1789760788.907243:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fcb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.907260:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760788.907264:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.907270:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fcb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27561984->27566079] (req 27561984->27566079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.909710:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fcb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27561984->27566079] (req 27561984->27566079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760788.949591:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27561984 -> 27566080 00010000:00010000:0.0:1789760788.949613:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fcb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27561984->27566079] (req 27561984->27566079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.949626:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fcb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27561984->27566079] (req 27561984->27566079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.949635:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fcb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27561984->27566079] (req 27561984->27566079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.949642:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760788.949648:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fcb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27561984->27566079] (req 27561984->27566079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.949659:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fcb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27561984->27566079] (req 27561984->27566079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.949701:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fcb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27561984->27566079] (req 27561984->27566079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.949708:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fcb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27561984->27566079] (req 27561984->27566079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.949800:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495945088:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42611us (42993us total) trans 4294974046 rc 0/0 00000100:00100000:0.0:1789760788.949811:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6758 00000100:00100000:0.0:1789760788.949821:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773178163320 time 773227889974 state 0x2 ref 0 00000001:00080000:1.0:1789760788.949903:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974046 is committed 00000100:00100000:2.0:1789760788.954923:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760788.959362:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495945216 00000100:00100000:0.0:1789760788.959375:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495945216 00000100:00100000:0.0:1789760788.959439:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773178163320 time 773227889974 state 0x0 ref 1 00000100:00100000:0.0:1789760788.959456:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773228150390 time 773227889974 state 0x1 ref 1 00000100:00100000:0.0:1789760788.959486:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773228150390 time 773280203160 state 0x0 ref 1 00000100:00100000:0.0:1789760788.959499:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495945216:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760788.959544:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27566080, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:0.0:1789760788.959553:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fd2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.959570:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760788.959575:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.959581:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fd2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27566080->27570175] (req 27566080->27570175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.959595:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fd2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27566080->27570175] (req 27566080->27570175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.002233:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974047 is committed 00002000:00010000:0.0:1789760789.002275:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27566080 -> 27570176 00010000:00010000:0.0:1789760789.002317:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fd2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27566080->27570175] (req 27566080->27570175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.002330:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fd2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27566080->27570175] (req 27566080->27570175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.002339:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fd2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27566080->27570175] (req 27566080->27570175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.002347:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760789.002352:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fd2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27566080->27570175] (req 27566080->27570175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.002361:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fd2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27566080->27570175] (req 27566080->27570175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.002386:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fd2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27566080->27570175] (req 27566080->27570175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.002399:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fd2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27566080->27570175] (req 27566080->27570175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.002489:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495945216:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42987us (47570us total) trans 4294974047 rc 0/0 00000100:00100000:0.0:1789760789.002499:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6759 00000100:00100000:0.0:1789760789.002508:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773228150390 time 773280203160 state 0x2 ref 0 00000100:00100000:2.0:1789760789.008167:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760789.008646:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495945344 00000100:00100000:0.0:1789760789.008736:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495945344 00000100:00100000:0.0:1789760789.008820:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773228150390 time 773280203160 state 0x0 ref 1 00000100:00100000:0.0:1789760789.008836:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773280463576 time 773280203160 state 0x1 ref 1 00000100:00100000:0.0:1789760789.008886:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773280463576 time 773329588347 state 0x0 ref 1 00000100:00100000:0.0:1789760789.008900:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495945344:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760789.008952:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27570176, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:0.0:1789760789.008962:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fd9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.008979:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760789.008984:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.008991:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fd9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27570176->27574271] (req 27570176->27574271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.009005:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fd9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27570176->27574271] (req 27570176->27574271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760789.010020:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760789.062499:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974048 is committed 00002000:00010000:3.0:1789760789.062773:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27570176 -> 27574272 00010000:00010000:3.0:1789760789.062791:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fd9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27570176->27574271] (req 27570176->27574271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.062804:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fd9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27570176->27574271] (req 27570176->27574271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.062813:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fd9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27570176->27574271] (req 27570176->27574271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.062819:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760789.062825:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fd9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27570176->27574271] (req 27570176->27574271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.062833:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fd9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27570176->27574271] (req 27570176->27574271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.062855:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fd9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27570176->27574271] (req 27570176->27574271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.062863:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354547fd9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27570176->27574271] (req 27570176->27574271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.062966:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495945344:12345-192.168.203.9@tcp:4:dd.0 Request processed in 54064us (54804us total) trans 4294974048 rc 0/0 00000100:00100000:3.0:1789760789.062976:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6760 00000100:00100000:3.0:1789760789.062987:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773280463576 time 773329588347 state 0x2 ref 0 00000100:00100000:1.0:1789760789.066733:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760789.066808:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495945472 00000100:00100000:0.0:1789760789.066816:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495945472 00000100:00100000:0.0:1789760789.066863:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773280463576 time 773329588347 state 0x0 ref 1 00000100:00100000:0.0:1789760789.066874:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773329848763 time 773329588347 state 0x1 ref 1 00000100:00100000:0.0:1789760789.066898:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773329848763 time 773387615448 state 0x0 ref 1 00000100:00100000:0.0:1789760789.066910:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495945472:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760789.066953:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27574272, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:0.0:1789760789.066960:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fe0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.066972:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760789.066975:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.066980:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fe0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27574272->27578367] (req 27574272->27578367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.066990:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fe0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27574272->27578367] (req 27574272->27578367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.104128:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974049 is committed 00002000:00010000:3.0:1789760789.104209:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27574272 -> 27578368 00010000:00010000:3.0:1789760789.104226:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fe0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27574272->27578367] (req 27574272->27578367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.104238:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fe0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27574272->27578367] (req 27574272->27578367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.104246:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fe0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27574272->27578367] (req 27574272->27578367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.104254:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760789.104258:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fe0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27574272->27578367] (req 27574272->27578367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.104267:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fe0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27574272->27578367] (req 27574272->27578367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.104354:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fe0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27574272->27578367] (req 27574272->27578367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.104363:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6934247/0x4936748354547fe0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27574272->27578367] (req 27574272->27578367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.104499:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495945472:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37586us (37769us total) trans 4294974049 rc 0/0 00000100:00100000:3.0:1789760789.104510:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6761 00000100:00100000:3.0:1789760789.104521:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773329848763 time 773387615448 state 0x2 ref 0 00000100:00100000:1.0:1789760789.110345:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760789.110431:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495945600 00000100:00100000:0.0:1789760789.110442:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495945600 00000100:00100000:0.0:1789760789.110494:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773329848763 time 773387615448 state 0x0 ref 1 00000100:00100000:0.0:1789760789.110509:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773387875864 time 773387615448 state 0x1 ref 1 00000100:00100000:0.0:1789760789.110539:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773387875864 time 773431256054 state 0x0 ref 1 00000100:00100000:0.0:1789760789.110552:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495945600:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760789.110611:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27578368, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:0.0:1789760789.110621:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547fe7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.110637:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760789.110641:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.110647:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547fe7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27578368->27582463] (req 27578368->27582463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.110660:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547fe7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27578368->27582463] (req 27578368->27582463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.150466:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974050 is committed 00002000:00010000:1.0:1789760789.150810:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27578368 -> 27582464 00010000:00010000:1.0:1789760789.150826:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547fe7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27578368->27582463] (req 27578368->27582463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.150839:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547fe7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27578368->27582463] (req 27578368->27582463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.150849:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547fe7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27578368->27582463] (req 27578368->27582463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.150857:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760789.150863:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547fe7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27578368->27582463] (req 27578368->27582463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.150874:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547fe7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27578368->27582463] (req 27578368->27582463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.150894:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547fe7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27578368->27582463] (req 27578368->27582463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.150903:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547fe7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27578368->27582463] (req 27578368->27582463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.151140:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495945600:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40585us (40800us total) trans 4294974050 rc 0/0 00000100:00100000:1.0:1789760789.151151:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6762 00000100:00100000:1.0:1789760789.151162:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773387875864 time 773431256054 state 0x2 ref 0 00000100:00100000:0.0:1789760789.158257:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760789.158400:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495945728 00000100:00100000:3.0:1789760789.158410:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495945728 00000100:00100000:3.0:1789760789.158466:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773387875864 time 773431256054 state 0x0 ref 1 00000100:00100000:3.0:1789760789.158482:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773431516470 time 773431256054 state 0x1 ref 1 00000100:00100000:3.0:1789760789.158545:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773431516470 time 773479262226 state 0x0 ref 1 00000100:00100000:3.0:1789760789.158557:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01374a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495945728:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760789.158617:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27582464, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:3.0:1789760789.158635:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547fee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.158650:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760789.158654:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.158660:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547fee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27582464->27586559] (req 27582464->27586559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.158839:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547fee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27582464->27586559] (req 27582464->27586559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760789.190311:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27582464 -> 27586560 00010000:00010000:3.0:1789760789.190332:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547fee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27582464->27586559] (req 27582464->27586559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.190345:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547fee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27582464->27586559] (req 27582464->27586559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.190354:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547fee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27582464->27586559] (req 27582464->27586559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.190360:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760789.190365:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547fee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27582464->27586559] (req 27582464->27586559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.190373:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547fee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27582464->27586559] (req 27582464->27586559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.190390:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547fee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27582464->27586559] (req 27582464->27586559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.190396:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354547fee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27582464->27586559] (req 27582464->27586559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.190484:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01374a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495945728:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31923us (32230us total) trans 4294974051 rc 0/0 00000100:00100000:3.0:1789760789.190494:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6763 00000100:00100000:3.0:1789760789.190504:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773431516470 time 773479262226 state 0x2 ref 0 00000001:00080000:0.0:1789760789.190651:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974051 is committed 00000100:00100000:3.0:1789760789.194394:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760789.194954:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495945856 00000100:00100000:0.0:1789760789.194965:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495945856 00000100:00100000:0.0:1789760789.195026:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773431516470 time 773479262226 state 0x0 ref 1 00000100:00100000:0.0:1789760789.195042:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773479522642 time 773479262226 state 0x1 ref 1 00000100:00100000:0.0:1789760789.195072:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773479522642 time 773515788642 state 0x0 ref 1 00000100:00100000:0.0:1789760789.195083:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495945856:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760789.195131:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27586560, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:0.0:1789760789.195140:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547ff5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.195157:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760789.195161:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.195168:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547ff5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27586560->27590655] (req 27586560->27590655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.195182:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547ff5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27586560->27590655] (req 27586560->27590655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.227750:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974052 is committed 00002000:00010000:3.0:1789760789.228362:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27586560 -> 27590656 00010000:00010000:3.0:1789760789.228379:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547ff5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27586560->27590655] (req 27586560->27590655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.228399:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547ff5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27586560->27590655] (req 27586560->27590655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.228410:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547ff5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27586560->27590655] (req 27586560->27590655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.228418:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760789.228423:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547ff5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27586560->27590655] (req 27586560->27590655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.228432:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547ff5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27586560->27590655] (req 27586560->27590655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.228453:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547ff5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27586560->27590655] (req 27586560->27590655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.228460:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x4936748354547ff5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27586560->27590655] (req 27586560->27590655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.228594:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495945856:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33507us (34204us total) trans 4294974052 rc 0/0 00000100:00100000:3.0:1789760789.228604:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6764 00000100:00100000:3.0:1789760789.228614:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773479522642 time 773515788642 state 0x2 ref 0 00000100:00100000:1.0:1789760789.233113:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760789.233230:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495945984 00000100:00100000:2.0:1789760789.233240:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495945984 00000100:00100000:2.0:1789760789.233392:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773479522642 time 773515788642 state 0x0 ref 1 00000100:00100000:2.0:1789760789.233407:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773516049058 time 773515788642 state 0x1 ref 1 00000100:00100000:2.0:1789760789.233472:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773516049058 time 773554188057 state 0x0 ref 1 00000100:00100000:2.0:1789760789.233485:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495945984:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760789.233536:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27590656, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:2.0:1789760789.233545:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ffc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.233560:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:2.0:1789760789.233564:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.233570:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ffc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27590656->27594751] (req 27590656->27594751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.233585:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ffc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27590656->27594751] (req 27590656->27594751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760789.265580:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27590656 -> 27594752 00010000:00010000:1.0:1789760789.265600:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ffc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27590656->27594751] (req 27590656->27594751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.265614:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ffc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27590656->27594751] (req 27590656->27594751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.265625:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ffc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27590656->27594751] (req 27590656->27594751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.265633:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760789.265638:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ffc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27590656->27594751] (req 27590656->27594751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.265647:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ffc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27590656->27594751] (req 27590656->27594751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.265692:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ffc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27590656->27594751] (req 27590656->27594751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.265702:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354547ffc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27590656->27594751] (req 27590656->27594751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.265801:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495945984:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32314us (32693us total) trans 4294974053 rc 0/0 00000100:00100000:1.0:1789760789.265812:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6765 00000100:00100000:1.0:1789760789.265822:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773516049058 time 773554188057 state 0x2 ref 0 00000001:00080000:3.0:1789760789.269951:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974053 is committed 00000100:00100000:1.0:1789760789.270725:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760789.270821:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495946112 00000100:00100000:3.0:1789760789.270830:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495946112 00000100:00100000:3.0:1789760789.270881:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773516049058 time 773554188057 state 0x0 ref 1 00000100:00100000:3.0:1789760789.270896:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773554448473 time 773554188057 state 0x1 ref 1 00000100:00100000:3.0:1789760789.270952:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773554448473 time 773591668536 state 0x0 ref 1 00000100:00100000:3.0:1789760789.270965:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495946112:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760789.271020:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27594752, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:3.0:1789760789.271030:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548003 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.271048:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760789.271052:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.271060:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548003 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27594752->27598847] (req 27594752->27598847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.271074:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548003 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27594752->27598847] (req 27594752->27598847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.278985:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789760789.279017:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f0276dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520317056:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760789.279084:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760789.281940:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520317056 00000100:00100000:0.0:1789760789.281955:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520317056 00000100:00100000:0.0:1789760789.282131:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 218 00000100:00100000:0.0:1789760789.282144:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520317056:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789760789.282185:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760789.282269:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520317056:12345-0@lo:13:osp-pre-1-0.0 Request processed in 128us (3184us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760789.282293:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 218 00000100:00100000:3.0:1789760789.282404:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f0276dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520317056:0@lo:13:osp-pre-1-0.0 00000001:00080000:1.0:1789760789.317899:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974054 is committed 00002000:00010000:3.0:1789760789.318081:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27594752 -> 27598848 00010000:00010000:3.0:1789760789.318098:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548003 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27594752->27598847] (req 27594752->27598847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.318111:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548003 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27594752->27598847] (req 27594752->27598847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.318119:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548003 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27594752->27598847] (req 27594752->27598847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.318127:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760789.318131:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548003 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27594752->27598847] (req 27594752->27598847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.318140:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548003 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27594752->27598847] (req 27594752->27598847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.318159:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548003 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27594752->27598847] (req 27594752->27598847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.318166:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548003 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27594752->27598847] (req 27594752->27598847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.318326:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495946112:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47359us (47666us total) trans 4294974054 rc 0/0 00000100:00100000:3.0:1789760789.318337:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6766 00000100:00100000:3.0:1789760789.318346:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773554448473 time 773591668536 state 0x2 ref 0 00000100:00100000:2.0:1789760789.323717:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760789.324146:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495946240 00000100:00100000:3.0:1789760789.324156:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495946240 00000100:00100000:3.0:1789760789.324212:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773554448473 time 773591668536 state 0x0 ref 1 00000100:00100000:3.0:1789760789.324227:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773591928952 time 773591668536 state 0x1 ref 1 00000100:00100000:2.0:1789760789.324503:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773591928952 time 773645217658 state 0x0 ref 1 00000100:00100000:2.0:1789760789.324519:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495946240:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760789.324567:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27598848, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:2.0:1789760789.324576:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454800a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.324591:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:2.0:1789760789.324595:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.324601:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454800a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27598848->27602943] (req 27598848->27602943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.324613:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454800a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27598848->27602943] (req 27598848->27602943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.343856:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760789.387349:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27598848 -> 27602944 00010000:00010000:1.0:1789760789.387372:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454800a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27598848->27602943] (req 27598848->27602943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.387386:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454800a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27598848->27602943] (req 27598848->27602943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.387394:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454800a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27598848->27602943] (req 27598848->27602943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.387401:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760789.387406:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454800a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27598848->27602943] (req 27598848->27602943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.387415:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454800a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27598848->27602943] (req 27598848->27602943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.387435:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454800a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27598848->27602943] (req 27598848->27602943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.387441:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x493674835454800a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27598848->27602943] (req 27598848->27602943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.387531:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495946240:12345-192.168.203.9@tcp:4:dd.0 Request processed in 63013us (63820us total) trans 4294974055 rc 0/0 00000100:00100000:1.0:1789760789.387542:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6767 00000100:00100000:1.0:1789760789.387551:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773591928952 time 773645217658 state 0x2 ref 0 00000001:00080000:1.0:1789760789.387590:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974055 is committed 00000100:00100000:1.0:1789760789.391994:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760789.392740:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495946368 00000100:00100000:2.0:1789760789.392750:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495946368 00000100:00100000:2.0:1789760789.392806:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773591928952 time 773645217658 state 0x0 ref 1 00000100:00100000:2.0:1789760789.392821:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773645478074 time 773645217658 state 0x1 ref 1 00000100:00100000:2.0:1789760789.392849:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773645478074 time 773713565442 state 0x0 ref 1 00000100:00100000:2.0:1789760789.392865:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495946368:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760789.392915:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27602944, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:2.0:1789760789.392925:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548011 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.392947:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:2.0:1789760789.392952:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.392960:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548011 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27602944->27607039] (req 27602944->27607039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.392973:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548011 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27602944->27607039] (req 27602944->27607039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.432971:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974056 is committed 00002000:00010000:3.0:1789760789.433623:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27602944 -> 27607040 00010000:00010000:3.0:1789760789.433640:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548011 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27602944->27607039] (req 27602944->27607039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.433653:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548011 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27602944->27607039] (req 27602944->27607039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.433677:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548011 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27602944->27607039] (req 27602944->27607039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.433693:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760789.433699:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548011 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27602944->27607039] (req 27602944->27607039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.433708:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548011 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27602944->27607039] (req 27602944->27607039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.433729:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548011 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27602944->27607039] (req 27602944->27607039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.433737:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548011 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27602944->27607039] (req 27602944->27607039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.433833:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495946368:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40966us (41844us total) trans 4294974056 rc 0/0 00000100:00100000:3.0:1789760789.433844:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6768 00000100:00100000:3.0:1789760789.433854:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773645478074 time 773713565442 state 0x2 ref 0 00000100:00100000:2.0:1789760789.438051:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760789.438137:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495946496 00000100:00100000:3.0:1789760789.438146:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495946496 00000100:00100000:3.0:1789760789.438212:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773645478074 time 773713565442 state 0x0 ref 1 00000100:00100000:3.0:1789760789.438227:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773713825858 time 773713565442 state 0x1 ref 1 00000100:00100000:3.0:1789760789.438255:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773713825858 time 773758971911 state 0x0 ref 1 00000100:00100000:3.0:1789760789.438267:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495946496:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760789.438720:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27607040, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:3.0:1789760789.438730:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548018 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.438745:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760789.438749:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.438755:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548018 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27607040->27611135] (req 27607040->27611135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.438768:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548018 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27607040->27611135] (req 27607040->27611135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.468549:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974057 is committed 00002000:00010000:3.0:1789760789.468752:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27607040 -> 27611136 00010000:00010000:3.0:1789760789.468769:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548018 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27607040->27611135] (req 27607040->27611135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.468782:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548018 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27607040->27611135] (req 27607040->27611135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.468791:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548018 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27607040->27611135] (req 27607040->27611135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.468799:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760789.468804:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548018 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27607040->27611135] (req 27607040->27611135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.468814:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548018 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27607040->27611135] (req 27607040->27611135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.468834:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548018 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27607040->27611135] (req 27607040->27611135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.468843:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548018 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27607040->27611135] (req 27607040->27611135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.468991:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495946496:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30721us (30945us total) trans 4294974057 rc 0/0 00000100:00100000:3.0:1789760789.469016:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6769 00000100:00100000:3.0:1789760789.469027:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773713825858 time 773758971911 state 0x2 ref 0 00000100:00100000:2.0:1789760789.473575:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760789.473743:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495946624 00000100:00100000:1.0:1789760789.473753:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495946624 00000100:00100000:1.0:1789760789.473809:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773713825858 time 773758971911 state 0x0 ref 1 00000100:00100000:1.0:1789760789.473825:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773759232327 time 773758971911 state 0x1 ref 1 00000100:00100000:1.0:1789760789.473891:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773759232327 time 773794607915 state 0x0 ref 1 00000100:00100000:1.0:1789760789.473905:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495946624:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760789.473957:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27611136, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:1.0:1789760789.473967:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454801f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.473984:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:1.0:1789760789.473989:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.473995:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454801f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27611136->27615231] (req 27611136->27615231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.474072:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454801f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27611136->27615231] (req 27611136->27615231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760789.506484:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27611136 -> 27615232 00010000:00010000:3.0:1789760789.506503:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454801f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27611136->27615231] (req 27611136->27615231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.506518:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454801f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27611136->27615231] (req 27611136->27615231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.506528:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454801f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27611136->27615231] (req 27611136->27615231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.506536:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760789.506542:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454801f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27611136->27615231] (req 27611136->27615231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.506551:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454801f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27611136->27615231] (req 27611136->27615231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.506571:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454801f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27611136->27615231] (req 27611136->27615231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.506580:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454801f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27611136->27615231] (req 27611136->27615231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.506692:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495946624:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32777us (33118us total) trans 4294974058 rc 0/0 00000100:00100000:3.0:1789760789.506703:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6770 00000100:00100000:3.0:1789760789.506712:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773759232327 time 773794607915 state 0x2 ref 0 00000001:00080000:0.0:1789760789.506835:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974058 is committed 00000100:00100000:1.0:1789760789.510577:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760789.510650:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495946752 00000100:00100000:3.0:1789760789.510668:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495946752 00000100:00100000:3.0:1789760789.510734:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773759232327 time 773794607915 state 0x0 ref 1 00000100:00100000:3.0:1789760789.510749:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773794868331 time 773794607915 state 0x1 ref 1 00000100:00100000:3.0:1789760789.510779:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773794868331 time 773831496441 state 0x0 ref 1 00000100:00100000:3.0:1789760789.510791:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495946752:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760789.510838:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27615232, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:3.0:1789760789.510848:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548026 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.510864:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760789.510869:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.510875:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548026 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27615232->27619327] (req 27615232->27619327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.510888:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548026 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27615232->27619327] (req 27615232->27619327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.558689:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974059 is committed 00002000:00010000:0.0:1789760789.559791:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27615232 -> 27619328 00010000:00010000:0.0:1789760789.559809:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548026 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27615232->27619327] (req 27615232->27619327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.559822:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548026 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27615232->27619327] (req 27615232->27619327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.559833:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548026 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27615232->27619327] (req 27615232->27619327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.559841:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760789.559846:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548026 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27615232->27619327] (req 27615232->27619327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.559856:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548026 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27615232->27619327] (req 27615232->27619327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.559879:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548026 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27615232->27619327] (req 27615232->27619327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.559888:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548026 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27615232->27619327] (req 27615232->27619327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.560115:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495946752:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49320us (49542us total) trans 4294974059 rc 0/0 00000100:00100000:0.0:1789760789.560127:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6771 00000100:00100000:0.0:1789760789.560136:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773794868331 time 773831496441 state 0x2 ref 0 00000100:00100000:1.0:1789760789.565791:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760789.565959:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495946880 00000100:00100000:3.0:1789760789.565970:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495946880 00000100:00100000:3.0:1789760789.566039:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773794868331 time 773831496441 state 0x0 ref 1 00000100:00100000:3.0:1789760789.566055:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773831756857 time 773831496441 state 0x1 ref 1 00000100:00100000:3.0:1789760789.566085:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773831756857 time 773886802383 state 0x0 ref 1 00000100:00100000:3.0:1789760789.566097:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495946880:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760789.566143:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27619328, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:3.0:1789760789.566152:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454802d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.566168:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760789.566173:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.566180:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454802d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27619328->27623423] (req 27619328->27623423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.566445:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454802d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27619328->27623423] (req 27619328->27623423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760789.606077:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27619328 -> 27623424 00010000:00010000:0.0:1789760789.606097:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454802d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27619328->27623423] (req 27619328->27623423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.606110:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454802d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27619328->27623423] (req 27619328->27623423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.606120:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454802d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27619328->27623423] (req 27619328->27623423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.606127:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760789.606132:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454802d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27619328->27623423] (req 27619328->27623423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.606142:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454802d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27619328->27623423] (req 27619328->27623423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.606343:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454802d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27619328->27623423] (req 27619328->27623423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.606356:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x493674835454802d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27619328->27623423] (req 27619328->27623423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.606487:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495946880:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40386us (40700us total) trans 4294974060 rc 0/0 00000100:00100000:0.0:1789760789.606498:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6772 00000100:00100000:0.0:1789760789.606507:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773831756857 time 773886802383 state 0x2 ref 0 00000001:00080000:1.0:1789760789.606510:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974060 is committed 00000100:00100000:2.0:1789760789.612982:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760789.613148:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495947008 00000100:00100000:3.0:1789760789.613158:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495947008 00000100:00100000:2.0:1789760789.613469:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773831756857 time 773886802383 state 0x0 ref 1 00000100:00100000:2.0:1789760789.613484:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773887062799 time 773886802383 state 0x1 ref 1 00000100:00100000:2.0:1789760789.613555:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773887062799 time 773934271761 state 0x0 ref 1 00000100:00100000:2.0:1789760789.613568:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04575500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495947008:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760789.613618:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27623424, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:2.0:1789760789.613627:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548034 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.613642:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:2.0:1789760789.613647:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.613653:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548034 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27623424->27627519] (req 27623424->27627519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.613677:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548034 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27623424->27627519] (req 27623424->27627519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.659994:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974061 is committed 00002000:00010000:2.0:1789760789.660144:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27623424 -> 27627520 00010000:00010000:2.0:1789760789.660159:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548034 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27623424->27627519] (req 27623424->27627519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.660172:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548034 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27623424->27627519] (req 27623424->27627519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.660182:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548034 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27623424->27627519] (req 27623424->27627519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.660190:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760789.660195:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548034 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27623424->27627519] (req 27623424->27627519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.660205:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548034 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27623424->27627519] (req 27623424->27627519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.660224:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548034 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27623424->27627519] (req 27623424->27627519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.660232:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548034 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27623424->27627519] (req 27623424->27627519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.660438:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04575500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495947008:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46866us (47459us total) trans 4294974061 rc 0/0 00000100:00100000:2.0:1789760789.660449:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6773 00000100:00100000:2.0:1789760789.660460:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773887062799 time 773934271761 state 0x2 ref 0 00000100:00100000:2.0:1789760789.663716:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760789.663803:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495947136 00000100:00100000:3.0:1789760789.663813:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495947136 00000100:00100000:3.0:1789760789.663865:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773887062799 time 773934271761 state 0x0 ref 1 00000100:00100000:3.0:1789760789.663880:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773934532177 time 773934271761 state 0x1 ref 1 00000100:00100000:3.0:1789760789.663910:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773934532177 time 773984627090 state 0x0 ref 1 00000100:00100000:3.0:1789760789.663923:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04577b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495947136:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760789.663969:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27627520, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:3.0:1789760789.663978:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454803b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.663994:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760789.663999:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.664018:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454803b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27627520->27631615] (req 27627520->27631615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.664031:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454803b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27627520->27631615] (req 27627520->27631615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.697372:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974062 is committed 00002000:00010000:0.0:1789760789.697786:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27627520 -> 27631616 00002000:00010000:0.0:1789760789.697791:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760788 -> 1789760789 00002000:00010000:0.0:1789760789.697794:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760788 -> 1789760789 00010000:00010000:0.0:1789760789.697809:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454803b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27627520->27631615] (req 27627520->27631615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.697821:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454803b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27627520->27631615] (req 27627520->27631615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.697830:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454803b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27627520->27631615] (req 27627520->27631615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.697838:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760789.697842:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454803b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27627520->27631615] (req 27627520->27631615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.697855:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454803b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27627520->27631615] (req 27627520->27631615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.697877:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454803b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27627520->27631615] (req 27627520->27631615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.697885:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454803b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27627520->27631615] (req 27627520->27631615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.698092:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04577b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495947136:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34166us (34380us total) trans 4294974062 rc 0/0 00000100:00100000:0.0:1789760789.698102:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6774 00000100:00100000:0.0:1789760789.698111:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773934532177 time 773984627090 state 0x2 ref 0 00000100:00100000:1.0:1789760789.702731:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760789.703421:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495947264 00000100:00100000:0.0:1789760789.703432:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495947264 00000100:00100000:1.0:1789760789.703535:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773934532177 time 773984627090 state 0x0 ref 1 00000100:00100000:1.0:1789760789.703550:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773984887506 time 773984627090 state 0x1 ref 1 00000100:00100000:1.0:1789760789.703583:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773984887506 time 774024299144 state 0x0 ref 1 00000100:00100000:1.0:1789760789.703596:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495947264:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760789.703646:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27631616, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:1.0:1789760789.703654:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354548042 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.703753:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760789.703757:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.703764:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354548042 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27631616->27635711] (req 27631616->27635711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.703779:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354548042 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27631616->27635711] (req 27631616->27635711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.735148:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974063 is committed 00002000:00010000:0.0:1789760789.735265:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27631616 -> 27635712 00010000:00010000:0.0:1789760789.735312:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354548042 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27631616->27635711] (req 27631616->27635711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.735325:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354548042 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27631616->27635711] (req 27631616->27635711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.735334:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354548042 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27631616->27635711] (req 27631616->27635711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.735345:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760789.735350:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354548042 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27631616->27635711] (req 27631616->27635711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.735358:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354548042 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27631616->27635711] (req 27631616->27635711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.735378:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354548042 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27631616->27635711] (req 27631616->27635711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.735385:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354548042 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27631616->27635711] (req 27631616->27635711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.735479:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495947264:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31871us (32742us total) trans 4294974063 rc 0/0 00000100:00100000:0.0:1789760789.735489:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6775 00000100:00100000:0.0:1789760789.735499:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773984887506 time 774024299144 state 0x2 ref 0 00000100:00100000:3.0:1789760789.740086:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760789.740238:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495947392 00000100:00100000:1.0:1789760789.740249:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495947392 00000100:00100000:1.0:1789760789.740347:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 773984887506 time 774024299144 state 0x0 ref 1 00000100:00100000:1.0:1789760789.740362:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774024559560 time 774024299144 state 0x1 ref 1 00000100:00100000:1.0:1789760789.740387:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774024559560 time 774061104606 state 0x0 ref 1 00000100:00100000:1.0:1789760789.740400:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495947392:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760789.740449:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27635712, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:1.0:1789760789.740458:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548049 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.740474:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760789.740478:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.740484:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548049 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27635712->27639807] (req 27635712->27639807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.740501:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548049 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27635712->27639807] (req 27635712->27639807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.800239:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974064 is committed 00002000:00010000:1.0:1789760789.800344:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27635712 -> 27639808 00010000:00010000:1.0:1789760789.800359:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548049 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27635712->27639807] (req 27635712->27639807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.800373:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548049 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27635712->27639807] (req 27635712->27639807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.800383:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548049 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27635712->27639807] (req 27635712->27639807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.800390:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760789.800396:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548049 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27635712->27639807] (req 27635712->27639807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.800405:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548049 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27635712->27639807] (req 27635712->27639807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.800426:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548049 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27635712->27639807] (req 27635712->27639807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.800434:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548049 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27635712->27639807] (req 27635712->27639807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.800665:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495947392:12345-192.168.203.9@tcp:4:dd.0 Request processed in 60255us (60580us total) trans 4294974064 rc 0/0 00000100:00100000:1.0:1789760789.800677:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6776 00000100:00100000:1.0:1789760789.800696:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774024559560 time 774061104606 state 0x2 ref 0 00000100:00100000:3.0:1789760789.808145:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760789.808261:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495947520 00000100:00100000:0.0:1789760789.808271:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495947520 00000100:00100000:0.0:1789760789.808381:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774024559560 time 774061104606 state 0x0 ref 1 00000100:00100000:0.0:1789760789.808400:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774061365022 time 774061104606 state 0x1 ref 1 00000100:00100000:0.0:1789760789.808436:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774061365022 time 774129152869 state 0x0 ref 1 00000100:00100000:0.0:1789760789.808450:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495947520:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760789.808514:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27639808, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:0.0:1789760789.808524:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548050 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.808543:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:0.0:1789760789.808549:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.808557:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548050 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27639808->27643903] (req 27639808->27643903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.808575:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548050 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27639808->27643903] (req 27639808->27643903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.842843:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974065 is committed 00002000:00010000:0.0:1789760789.842934:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27639808 -> 27643904 00010000:00010000:0.0:1789760789.842950:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548050 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27639808->27643903] (req 27639808->27643903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.842963:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548050 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27639808->27643903] (req 27639808->27643903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.842972:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548050 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27639808->27643903] (req 27639808->27643903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.842979:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760789.842984:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548050 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27639808->27643903] (req 27639808->27643903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.842993:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548050 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27639808->27643903] (req 27639808->27643903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.843012:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548050 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27639808->27643903] (req 27639808->27643903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.843019:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548050 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27639808->27643903] (req 27639808->27643903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.843631:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495947520:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35178us (35492us total) trans 4294974065 rc 0/0 00000100:00100000:0.0:1789760789.843642:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6777 00000100:00100000:0.0:1789760789.843653:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774061365022 time 774129152869 state 0x2 ref 0 00000100:00100000:2.0:1789760789.851091:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760789.851197:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495947648 00000100:00100000:0.0:1789760789.851208:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495947648 00000100:00100000:0.0:1789760789.851264:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774061365022 time 774129152869 state 0x0 ref 1 00000100:00100000:0.0:1789760789.851310:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774129413285 time 774129152869 state 0x1 ref 1 00000100:00100000:0.0:1789760789.851342:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774129413285 time 774172058983 state 0x0 ref 1 00000100:00100000:0.0:1789760789.851355:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04929500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495947648:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760789.851404:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27643904, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:0.0:1789760789.851414:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354548057 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.851431:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:0.0:1789760789.851436:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.851442:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354548057 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27643904->27647999] (req 27643904->27647999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.851456:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354548057 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27643904->27647999] (req 27643904->27647999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.906456:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974066 is committed 00002000:00010000:2.0:1789760789.906570:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27643904 -> 27648000 00010000:00010000:2.0:1789760789.906585:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354548057 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27643904->27647999] (req 27643904->27647999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.906598:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354548057 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27643904->27647999] (req 27643904->27647999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.906606:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354548057 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27643904->27647999] (req 27643904->27647999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.906613:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760789.906618:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354548057 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27643904->27647999] (req 27643904->27647999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.906631:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354548057 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27643904->27647999] (req 27643904->27647999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.906650:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354548057 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27643904->27647999] (req 27643904->27647999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.906669:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x4936748354548057 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27643904->27647999] (req 27643904->27647999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.906769:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04929500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495947648:12345-192.168.203.9@tcp:4:dd.0 Request processed in 55413us (55686us total) trans 4294974066 rc 0/0 00000100:00100000:2.0:1789760789.906780:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6778 00000100:00100000:2.0:1789760789.906790:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774129413285 time 774172058983 state 0x2 ref 0 00000100:00100000:3.0:1789760789.911046:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760789.911194:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495947776 00000100:00100000:1.0:1789760789.911205:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495947776 00000100:00100000:1.0:1789760789.911259:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774129413285 time 774172058983 state 0x0 ref 1 00000100:00100000:1.0:1789760789.911275:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774172319399 time 774172058983 state 0x1 ref 1 00000100:00100000:1.0:1789760789.911345:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774172319399 time 774232061375 state 0x0 ref 1 00000100:00100000:1.0:1789760789.911357:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495947776:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760789.911407:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27648000, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:1.0:1789760789.911416:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454805e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.911432:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760789.911436:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.911443:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454805e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27648000->27652095] (req 27648000->27652095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.911457:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454805e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27648000->27652095] (req 27648000->27652095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.944935:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974067 is committed 00002000:00010000:1.0:1789760789.945101:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27648000 -> 27652096 00010000:00010000:1.0:1789760789.945127:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454805e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27648000->27652095] (req 27648000->27652095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.945141:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454805e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27648000->27652095] (req 27648000->27652095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.945150:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454805e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27648000->27652095] (req 27648000->27652095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.945158:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760789.945162:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454805e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27648000->27652095] (req 27648000->27652095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.945171:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454805e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27648000->27652095] (req 27648000->27652095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.945191:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454805e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27648000->27652095] (req 27648000->27652095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.945198:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454805e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27648000->27652095] (req 27648000->27652095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.945402:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495947776:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34042us (34360us total) trans 4294974067 rc 0/0 00000100:00100000:1.0:1789760789.945416:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6779 00000100:00100000:1.0:1789760789.945425:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774172319399 time 774232061375 state 0x2 ref 0 00000100:00100000:3.0:1789760789.949147:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760789.949413:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495947904 00000100:00100000:0.0:1789760789.949423:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495947904 00000100:00100000:0.0:1789760789.949477:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774172319399 time 774232061375 state 0x0 ref 1 00000100:00100000:0.0:1789760789.949493:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774232321791 time 774232061375 state 0x1 ref 1 00000100:00100000:0.0:1789760789.949554:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774232321791 time 774270271041 state 0x0 ref 1 00000100:00100000:0.0:1789760789.949568:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495947904:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760789.949618:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27652096, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:0.0:1789760789.949628:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548065 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.949645:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:0.0:1789760789.949649:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.949655:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548065 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27652096->27656191] (req 27652096->27656191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.949670:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548065 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27652096->27656191] (req 27652096->27656191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.982157:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974068 is committed 00002000:00010000:1.0:1789760789.982414:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27652096 -> 27656192 00010000:00010000:1.0:1789760789.982434:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548065 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27652096->27656191] (req 27652096->27656191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.982447:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548065 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27652096->27656191] (req 27652096->27656191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.982457:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548065 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27652096->27656191] (req 27652096->27656191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.982463:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760789.982468:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548065 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27652096->27656191] (req 27652096->27656191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.982478:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548065 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27652096->27656191] (req 27652096->27656191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.982497:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548065 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27652096->27656191] (req 27652096->27656191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.982504:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548065 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27652096->27656191] (req 27652096->27656191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.982710:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495947904:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33027us (33457us total) trans 4294974068 rc 0/0 00000100:00100000:1.0:1789760789.982721:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6780 00000100:00100000:1.0:1789760789.982731:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774232321791 time 774270271041 state 0x2 ref 0 00000100:00100000:2.0:1789760789.992647:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760789.992928:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495948032 00000100:00100000:0.0:1789760789.992938:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495948032 00000100:00100000:0.0:1789760789.992990:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774232321791 time 774270271041 state 0x0 ref 1 00000100:00100000:0.0:1789760789.993004:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774270531457 time 774270271041 state 0x1 ref 1 00000100:00100000:0.0:1789760789.993071:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774270531457 time 774313787791 state 0x0 ref 1 00000100:00100000:0.0:1789760789.993084:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495948032:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760789.993140:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27656192, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:0.0:1789760789.993148:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454806c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.993165:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:0.0:1789760789.993169:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.993176:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454806c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27656192->27660287] (req 27656192->27660287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.993189:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454806c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27656192->27660287] (req 27656192->27660287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.006927:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760790.007192:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760790.007235:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760790.007326:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495948160 00000100:00100000:3.0:1789760790.007338:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495948160 00000100:00100000:1.0:1789760790.007388:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495948288 00000100:00100000:1.0:1789760790.007394:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495948288 00000100:00100000:3.0:1789760790.007401:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 420 00000100:00100000:3.0:1789760790.007413:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290122cc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2358:x1876699495948160:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:3.0:1789760790.007508:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290122cc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2358:x1876699495948160:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 97us (317us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760790.007511:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495948416 00000100:00100000:1.0:1789760790.007517:0:5853:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495948416 00000100:00100000:3.0:1789760790.007520:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 420 00000100:00100000:1.0:1789760790.007548:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 530 00000100:00100000:1.0:1789760790.007558:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03096680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:17022965-1e4f-46fb-915c-61730096b8e2+8:2358:x1876699495948288:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:1.0:1789760790.007594:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03096680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:17022965-1e4f-46fb-915c-61730096b8e2+8:2358:x1876699495948288:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 39us (678us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760790.007600:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 530 00000100:00100000:1.0:1789760790.007612:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 531 00000100:00080000:3.0:1789760790.008494:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760790.010199:0:5853:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92901237f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2358:x1876699495948416:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:1.0:1789760790.010396:0:5853:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92901237f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2358:x1876699495948416:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 2776us (3158us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760790.010407:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 531 00002000:00010000:1.0:1789760790.041380:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27656192 -> 27660288 00010000:00010000:1.0:1789760790.041404:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454806c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27656192->27660287] (req 27656192->27660287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.041417:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454806c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27656192->27660287] (req 27656192->27660287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.041427:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454806c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27656192->27660287] (req 27656192->27660287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.041435:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760790.041441:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454806c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27656192->27660287] (req 27656192->27660287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.041451:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454806c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27656192->27660287] (req 27656192->27660287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.041475:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454806c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27656192->27660287] (req 27656192->27660287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.041483:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454806c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27656192->27660287] (req 27656192->27660287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.041505:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974069 is committed 00000100:00100000:1.0:1789760790.041580:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495948032:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48493us (48938us total) trans 4294974069 rc 0/0 00000100:00100000:1.0:1789760790.041591:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6781 00000100:00100000:1.0:1789760790.041602:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774270531457 time 774313787791 state 0x2 ref 0 00000100:00100000:2.0:1789760790.045369:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760790.047567:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495948544 00000100:00100000:1.0:1789760790.047584:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495948544 00000100:00100000:1.0:1789760790.047684:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774270531457 time 774313787791 state 0x0 ref 1 00000100:00100000:1.0:1789760790.047702:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774314048207 time 774313787791 state 0x1 ref 1 00000100:00100000:1.0:1789760790.047734:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774314048207 time 774368451502 state 0x0 ref 1 00000100:00100000:1.0:1789760790.047756:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495948544:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760790.047810:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27660288, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:1.0:1789760790.047819:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548073 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.047834:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760790.047838:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.047845:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548073 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27660288->27664383] (req 27660288->27664383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.047857:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548073 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27660288->27664383] (req 27660288->27664383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760790.087590:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27660288 -> 27664384 00000001:00080000:0.0:1789760790.087614:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974070 is committed 00010000:00010000:1.0:1789760790.087624:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548073 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27660288->27664383] (req 27660288->27664383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.087638:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548073 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27660288->27664383] (req 27660288->27664383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.087649:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548073 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27660288->27664383] (req 27660288->27664383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.087657:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760790.087663:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548073 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27660288->27664383] (req 27660288->27664383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.087672:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548073 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27660288->27664383] (req 27660288->27664383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.087700:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548073 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27660288->27664383] (req 27660288->27664383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.087709:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548073 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27660288->27664383] (req 27660288->27664383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.087841:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495948544:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40084us (42480us total) trans 4294974070 rc 0/0 00000100:00100000:1.0:1789760790.087849:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6782 00000100:00100000:1.0:1789760790.087858:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774314048207 time 774368451502 state 0x2 ref 0 00000100:00100000:2.0:1789760790.092596:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760790.092802:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495948672 00000100:00100000:1.0:1789760790.092812:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495948672 00000100:00100000:1.0:1789760790.092867:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774314048207 time 774368451502 state 0x0 ref 1 00000100:00100000:1.0:1789760790.092881:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774368711918 time 774368451502 state 0x1 ref 1 00000100:00100000:1.0:1789760790.092911:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774368711918 time 774413627917 state 0x0 ref 1 00000100:00100000:1.0:1789760790.092929:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495948672:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760790.092977:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27664384, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:1.0:1789760790.092987:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454807a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.093004:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760790.093008:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.093014:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454807a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27664384->27668479] (req 27664384->27668479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.093080:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454807a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27664384->27668479] (req 27664384->27668479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.145517:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974071 is committed 00002000:00010000:1.0:1789760790.145569:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27664384 -> 27668480 00010000:00010000:1.0:1789760790.145585:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454807a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27664384->27668479] (req 27664384->27668479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.145598:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454807a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27664384->27668479] (req 27664384->27668479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.145606:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454807a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27664384->27668479] (req 27664384->27668479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.145613:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760790.145617:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454807a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27664384->27668479] (req 27664384->27668479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.145626:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454807a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27664384->27668479] (req 27664384->27668479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.145660:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454807a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27664384->27668479] (req 27664384->27668479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.145668:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454807a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27664384->27668479] (req 27664384->27668479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.145783:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495948672:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52857us (53193us total) trans 4294974071 rc 0/0 00000100:00100000:1.0:1789760790.145793:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6783 00000100:00100000:1.0:1789760790.145810:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774368711918 time 774413627917 state 0x2 ref 0 00000100:00100000:2.0:1789760790.150418:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760790.150501:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495948800 00000100:00100000:1.0:1789760790.150512:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495948800 00000100:00100000:1.0:1789760790.150570:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774368711918 time 774413627917 state 0x0 ref 1 00000100:00100000:1.0:1789760790.150585:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774413888333 time 774413627917 state 0x1 ref 1 00000100:00100000:1.0:1789760790.150619:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774413888333 time 774471333970 state 0x0 ref 1 00000100:00100000:1.0:1789760790.150642:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495948800:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760790.150713:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27668480, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:1.0:1789760790.150722:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548081 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.150739:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760790.150745:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.150751:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548081 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27668480->27672575] (req 27668480->27672575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.150765:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548081 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27668480->27672575] (req 27668480->27672575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760790.189055:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27668480 -> 27672576 00010000:00010000:1.0:1789760790.189085:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548081 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27668480->27672575] (req 27668480->27672575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.189098:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548081 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27668480->27672575] (req 27668480->27672575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.189115:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548081 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27668480->27672575] (req 27668480->27672575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.189123:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760790.189128:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548081 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27668480->27672575] (req 27668480->27672575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.189138:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548081 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27668480->27672575] (req 27668480->27672575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.189158:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548081 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27668480->27672575] (req 27668480->27672575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.189167:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548081 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27668480->27672575] (req 27668480->27672575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.189275:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495948800:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38631us (38864us total) trans 4294974072 rc 0/0 00000100:00100000:1.0:1789760790.189330:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6784 00000100:00100000:1.0:1789760790.189340:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774413888333 time 774471333970 state 0x2 ref 0 00000001:00080000:2.0:1789760790.189399:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974072 is committed 00000100:00100000:2.0:1789760790.192515:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760790.192610:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495948928 00000100:00100000:1.0:1789760790.192620:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495948928 00000100:00100000:1.0:1789760790.192692:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774413888333 time 774471333970 state 0x0 ref 1 00000100:00100000:1.0:1789760790.192709:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774471594386 time 774471333970 state 0x1 ref 1 00000100:00100000:1.0:1789760790.192736:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774471594386 time 774513452926 state 0x0 ref 1 00000100:00100000:1.0:1789760790.192748:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01c8d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495948928:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760790.192797:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27672576, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:1.0:1789760790.192806:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548088 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.192821:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760790.192825:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.192832:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548088 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27672576->27676671] (req 27672576->27676671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.192846:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548088 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27672576->27676671] (req 27672576->27676671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760790.233206:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27672576 -> 27676672 00000001:00080000:2.0:1789760790.233318:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974073 is committed 00010000:00010000:1.0:1789760790.233341:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548088 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27672576->27676671] (req 27672576->27676671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.233355:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548088 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27672576->27676671] (req 27672576->27676671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.233364:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548088 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27672576->27676671] (req 27672576->27676671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.233372:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760790.233377:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548088 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27672576->27676671] (req 27672576->27676671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.233387:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548088 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27672576->27676671] (req 27672576->27676671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.233406:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548088 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27672576->27676671] (req 27672576->27676671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.233413:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548088 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27672576->27676671] (req 27672576->27676671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.233494:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01c8d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495948928:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40744us (40984us total) trans 4294974073 rc 0/0 00000100:00100000:1.0:1789760790.233509:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6785 00000100:00100000:1.0:1789760790.233519:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774471594386 time 774513452926 state 0x2 ref 0 00000100:00100000:2.0:1789760790.236931:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760790.237161:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495949056 00000100:00100000:1.0:1789760790.237170:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495949056 00000100:00100000:1.0:1789760790.237405:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774471594386 time 774513452926 state 0x0 ref 1 00000100:00100000:1.0:1789760790.237426:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774513713342 time 774513452926 state 0x1 ref 1 00000100:00100000:1.0:1789760790.237503:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774513713342 time 774558218593 state 0x0 ref 1 00000100:00100000:1.0:1789760790.237516:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fdca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495949056:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760790.237562:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27676672, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:1.0:1789760790.237571:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454808f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.237586:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760790.237591:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.237597:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454808f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27676672->27680767] (req 27676672->27680767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.237612:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454808f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27676672->27680767] (req 27676672->27680767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.274751:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974074 is committed 00002000:00010000:1.0:1789760790.275029:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27676672 -> 27680768 00010000:00010000:1.0:1789760790.275045:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454808f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27676672->27680767] (req 27676672->27680767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.275059:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454808f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27676672->27680767] (req 27676672->27680767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.275068:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454808f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27676672->27680767] (req 27676672->27680767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.275076:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760790.275081:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454808f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27676672->27680767] (req 27676672->27680767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.275089:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454808f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27676672->27680767] (req 27676672->27680767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.275173:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454808f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27676672->27680767] (req 27676672->27680767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.275182:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x493674835454808f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27676672->27680767] (req 27676672->27680767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.275427:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fdca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495949056:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37908us (38499us total) trans 4294974074 rc 0/0 00000100:00100000:1.0:1789760790.275438:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6786 00000100:00100000:1.0:1789760790.275447:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774513713342 time 774558218593 state 0x2 ref 0 00000100:00100000:2.0:1789760790.279208:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760790.279487:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495949184 00000100:00100000:1.0:1789760790.279497:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495949184 00000100:00100000:1.0:1789760790.279610:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774513713342 time 774558218593 state 0x0 ref 1 00000100:00100000:1.0:1789760790.279803:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774558479009 time 774558218593 state 0x1 ref 1 00000100:00100000:1.0:1789760790.279876:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774558479009 time 774600591386 state 0x0 ref 1 00000100:00100000:1.0:1789760790.279890:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495949184:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760790.279940:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27680768, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:1.0:1789760790.279950:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548096 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.279965:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760790.279970:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.279976:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548096 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27680768->27684863] (req 27680768->27684863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.279990:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548096 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27680768->27684863] (req 27680768->27684863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.313246:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974075 is committed 00002000:00010000:3.0:1789760790.313349:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27680768 -> 27684864 00010000:00010000:3.0:1789760790.313364:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548096 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27680768->27684863] (req 27680768->27684863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.313377:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548096 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27680768->27684863] (req 27680768->27684863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.313388:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548096 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27680768->27684863] (req 27680768->27684863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.313395:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760790.313399:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548096 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27680768->27684863] (req 27680768->27684863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.313406:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548096 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27680768->27684863] (req 27680768->27684863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.313421:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548096 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27680768->27684863] (req 27680768->27684863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.313430:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x4936748354548096 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27680768->27684863] (req 27680768->27684863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.313507:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495949184:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33617us (34305us total) trans 4294974075 rc 0/0 00000100:00100000:3.0:1789760790.313514:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6787 00000100:00100000:3.0:1789760790.313522:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774558479009 time 774600591386 state 0x2 ref 0 00000100:00100000:2.0:1789760790.317623:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760790.318107:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495949312 00000100:00100000:1.0:1789760790.318116:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495949312 00000100:00100000:1.0:1789760790.318170:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774558479009 time 774600591386 state 0x0 ref 1 00000100:00100000:1.0:1789760790.318184:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774600851802 time 774600591386 state 0x1 ref 1 00000100:00100000:1.0:1789760790.318214:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774600851802 time 774638930356 state 0x0 ref 1 00000100:00100000:1.0:1789760790.318230:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495949312:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760790.318299:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27684864, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:1.0:1789760790.318308:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454809d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.318325:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760790.318329:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.318337:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454809d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27684864->27688959] (req 27684864->27688959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.318351:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454809d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27684864->27688959] (req 27684864->27688959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.348968:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974076 is committed 00002000:00010000:3.0:1789760790.349784:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27684864 -> 27688960 00010000:00010000:3.0:1789760790.349800:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454809d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27684864->27688959] (req 27684864->27688959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.349813:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454809d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27684864->27688959] (req 27684864->27688959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.349823:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454809d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27684864->27688959] (req 27684864->27688959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.349830:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760790.349835:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454809d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27684864->27688959] (req 27684864->27688959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.349844:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454809d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27684864->27688959] (req 27684864->27688959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.349869:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454809d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27684864->27688959] (req 27684864->27688959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.349877:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x493674835454809d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27684864->27688959] (req 27684864->27688959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.349970:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495949312:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31737us (32353us total) trans 4294974076 rc 0/0 00000100:00100000:3.0:1789760790.349980:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6788 00000100:00100000:3.0:1789760790.349990:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774600851802 time 774638930356 state 0x2 ref 0 00000100:00100000:2.0:1789760790.362798:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760790.362898:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495949440 00000100:00100000:1.0:1789760790.362908:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495949440 00000100:00100000:1.0:1789760790.362964:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774600851802 time 774638930356 state 0x0 ref 1 00000100:00100000:1.0:1789760790.362980:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774639190772 time 774638930356 state 0x1 ref 1 00000100:00100000:1.0:1789760790.363010:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774639190772 time 774683726648 state 0x0 ref 1 00000100:00100000:1.0:1789760790.363024:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495949440:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760790.363076:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27688960, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:1.0:1789760790.363085:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545480a4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.363107:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760790.363111:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.363118:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545480a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27688960->27693055] (req 27688960->27693055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.363131:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545480a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27688960->27693055] (req 27688960->27693055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.368204:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760790.394493:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27688960 -> 27693056 00010000:00010000:0.0:1789760790.394515:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545480a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27688960->27693055] (req 27688960->27693055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.394528:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545480a4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27688960->27693055] (req 27688960->27693055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.394537:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545480a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27688960->27693055] (req 27688960->27693055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.394544:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760790.394627:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545480a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27688960->27693055] (req 27688960->27693055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.394639:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545480a4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27688960->27693055] (req 27688960->27693055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.394659:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545480a4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27688960->27693055] (req 27688960->27693055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.394667:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x49367483545480a4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27688960->27693055] (req 27688960->27693055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.394780:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495949440:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31754us (31985us total) trans 4294974077 rc 0/0 00000100:00100000:0.0:1789760790.394791:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6789 00000100:00100000:0.0:1789760790.394801:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774639190772 time 774683726648 state 0x2 ref 0 00000001:00080000:2.0:1789760790.394944:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974077 is committed 00000100:00100000:2.0:1789760790.398525:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760790.398617:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495949568 00000100:00100000:3.0:1789760790.398628:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495949568 00000100:00100000:3.0:1789760790.398795:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774639190772 time 774683726648 state 0x0 ref 1 00000100:00100000:3.0:1789760790.398871:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774683987064 time 774683726648 state 0x1 ref 1 00000100:00100000:3.0:1789760790.398988:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774683987064 time 774719704162 state 0x0 ref 1 00000100:00100000:3.0:1789760790.399002:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495949568:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760790.399051:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27693056, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:3.0:1789760790.399060:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545480ab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.399076:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:3.0:1789760790.399086:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.399092:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545480ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27693056->27697151] (req 27693056->27697151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.399107:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545480ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27693056->27697151] (req 27693056->27697151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760790.427422:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27693056 -> 27697152 00010000:00010000:3.0:1789760790.427439:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545480ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27693056->27697151] (req 27693056->27697151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.427452:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545480ab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27693056->27697151] (req 27693056->27697151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.427462:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545480ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27693056->27697151] (req 27693056->27697151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.427469:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760790.427474:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545480ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27693056->27697151] (req 27693056->27697151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.427484:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545480ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27693056->27697151] (req 27693056->27697151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.427503:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545480ab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27693056->27697151] (req 27693056->27697151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.427511:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545480ab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27693056->27697151] (req 27693056->27697151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.427604:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495949568:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28601us (29084us total) trans 4294974078 rc 0/0 00000100:00100000:3.0:1789760790.427615:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6790 00000100:00100000:3.0:1789760790.427624:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774683987064 time 774719704162 state 0x2 ref 0 00000001:00080000:3.0:1789760790.427673:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974078 is committed 00000100:00100000:3.0:1789760790.431872:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760790.431947:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495949696 00000100:00100000:1.0:1789760790.431957:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495949696 00000100:00100000:1.0:1789760790.432009:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774683987064 time 774719704162 state 0x0 ref 1 00000100:00100000:1.0:1789760790.432024:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774719964578 time 774719704162 state 0x1 ref 1 00000100:00100000:1.0:1789760790.432106:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774719964578 time 774752822646 state 0x0 ref 1 00000100:00100000:1.0:1789760790.432119:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495949696:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760790.432176:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27697152, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:1.0:1789760790.432185:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545480b2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.432200:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760790.432204:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.432211:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545480b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27697152->27701247] (req 27697152->27701247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.432228:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545480b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27697152->27701247] (req 27697152->27701247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.480559:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974079 is committed 00002000:00010000:3.0:1789760790.480851:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27697152 -> 27701248 00010000:00010000:3.0:1789760790.480867:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545480b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27697152->27701247] (req 27697152->27701247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.480880:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545480b2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27697152->27701247] (req 27697152->27701247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.480890:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545480b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27697152->27701247] (req 27697152->27701247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.480898:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760790.480903:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545480b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27697152->27701247] (req 27697152->27701247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.480912:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545480b2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27697152->27701247] (req 27697152->27701247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.480933:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545480b2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27697152->27701247] (req 27697152->27701247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.480941:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545480b2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27697152->27701247] (req 27697152->27701247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.481142:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495949696:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49019us (49273us total) trans 4294974079 rc 0/0 00000100:00100000:3.0:1789760790.481153:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6791 00000100:00100000:3.0:1789760790.481163:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774719964578 time 774752822646 state 0x2 ref 0 00000100:00100000:2.0:1789760790.485048:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760790.485196:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495949824 00000100:00100000:0.0:1789760790.485207:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495949824 00000100:00100000:0.0:1789760790.485264:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774719964578 time 774752822646 state 0x0 ref 1 00000100:00100000:0.0:1789760790.485321:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774753083062 time 774752822646 state 0x1 ref 1 00000100:00100000:0.0:1789760790.485350:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774753083062 time 774806067394 state 0x0 ref 1 00000100:00100000:0.0:1789760790.485363:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495949824:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760790.485412:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27701248, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:0.0:1789760790.485421:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545480b9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.485436:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:0.0:1789760790.485440:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.485447:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545480b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27701248->27705343] (req 27701248->27705343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.485461:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545480b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27701248->27705343] (req 27701248->27705343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.528014:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974080 is committed 00002000:00010000:1.0:1789760790.528269:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27701248 -> 27705344 00010000:00010000:1.0:1789760790.528312:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545480b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27701248->27705343] (req 27701248->27705343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.528325:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545480b9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27701248->27705343] (req 27701248->27705343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.528333:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545480b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27701248->27705343] (req 27701248->27705343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.528340:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760790.528344:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545480b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27701248->27705343] (req 27701248->27705343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.528352:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545480b9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27701248->27705343] (req 27701248->27705343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.528372:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545480b9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27701248->27705343] (req 27701248->27705343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.528379:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x49367483545480b9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27701248->27705343] (req 27701248->27705343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.528467:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495949824:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43103us (43426us total) trans 4294974080 rc 0/0 00000100:00100000:1.0:1789760790.528477:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6792 00000100:00100000:1.0:1789760790.528486:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774753083062 time 774806067394 state 0x2 ref 0 00000100:00100000:2.0:1789760790.532087:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760790.532182:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495949952 00000100:00100000:3.0:1789760790.532192:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495949952 00000100:00100000:3.0:1789760790.532246:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774753083062 time 774806067394 state 0x0 ref 1 00000100:00100000:3.0:1789760790.532261:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774806327810 time 774806067394 state 0x1 ref 1 00000100:00100000:3.0:1789760790.532329:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774806327810 time 774853045810 state 0x0 ref 1 00000100:00100000:3.0:1789760790.532342:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495949952:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760790.532386:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27705344, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:3.0:1789760790.532395:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545480c0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.532410:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:3.0:1789760790.532414:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.532420:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545480c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27705344->27709439] (req 27705344->27709439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.532437:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545480c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27705344->27709439] (req 27705344->27709439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.587332:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974081 is committed 00002000:00010000:2.0:1789760790.590734:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27705344 -> 27709440 00010000:00010000:2.0:1789760790.590754:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545480c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27705344->27709439] (req 27705344->27709439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.590768:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545480c0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27705344->27709439] (req 27705344->27709439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.590777:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545480c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27705344->27709439] (req 27705344->27709439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.590785:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760790.590791:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545480c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27705344->27709439] (req 27705344->27709439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.590800:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545480c0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27705344->27709439] (req 27705344->27709439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.590826:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545480c0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27705344->27709439] (req 27705344->27709439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.590835:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545480c0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27705344->27709439] (req 27705344->27709439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.590962:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495949952:12345-192.168.203.9@tcp:4:dd.0 Request processed in 58611us (58872us total) trans 4294974081 rc 0/0 00000100:00100000:2.0:1789760790.590973:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6793 00000100:00100000:2.0:1789760790.590983:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774806327810 time 774853045810 state 0x2 ref 0 00000100:00100000:2.0:1789760790.594461:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760790.594517:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495950080 00000100:00100000:3.0:1789760790.594527:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495950080 00000100:00100000:3.0:1789760790.594576:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774806327810 time 774853045810 state 0x0 ref 1 00000100:00100000:3.0:1789760790.594589:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774853306226 time 774853045810 state 0x1 ref 1 00000100:00100000:3.0:1789760790.594613:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774853306226 time 774915330195 state 0x0 ref 1 00000100:00100000:3.0:1789760790.594625:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fdc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495950080:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760790.594683:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27709440, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:3.0:1789760790.594691:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545480c7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.594705:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:3.0:1789760790.594710:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.594716:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545480c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27709440->27713535] (req 27709440->27713535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.594730:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545480c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27709440->27713535] (req 27709440->27713535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760790.630994:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27709440 -> 27713536 00010000:00010000:0.0:1789760790.631014:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545480c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27709440->27713535] (req 27709440->27713535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.631027:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545480c7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27709440->27713535] (req 27709440->27713535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.631036:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545480c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27709440->27713535] (req 27709440->27713535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.631044:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760790.631048:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545480c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27709440->27713535] (req 27709440->27713535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.631057:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545480c7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27709440->27713535] (req 27709440->27713535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.631076:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545480c7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27709440->27713535] (req 27709440->27713535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.631084:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545480c7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27709440->27713535] (req 27709440->27713535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.631171:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fdc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495950080:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36543us (36714us total) trans 4294974082 rc 0/0 00000100:00100000:0.0:1789760790.631181:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6794 00000100:00100000:0.0:1789760790.631191:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774853306226 time 774915330195 state 0x2 ref 0 00000001:00080000:0.0:1789760790.631228:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974082 is committed 00000100:00100000:0.0:1789760790.634748:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760790.635187:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495950208 00000100:00100000:1.0:1789760790.635197:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495950208 00000100:00100000:1.0:1789760790.635252:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774853306226 time 774915330195 state 0x0 ref 1 00000100:00100000:1.0:1789760790.635267:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774915590611 time 774915330195 state 0x1 ref 1 00000100:00100000:1.0:1789760790.635331:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774915590611 time 774956047182 state 0x0 ref 1 00000100:00100000:1.0:1789760790.635344:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01375180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495950208:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760790.635393:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27713536, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:1.0:1789760790.635402:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545480ce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.635417:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760790.635422:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.635428:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545480ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27713536->27717631] (req 27713536->27717631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.635700:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545480ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27713536->27717631] (req 27713536->27717631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.686386:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974083 is committed 00002000:00010000:1.0:1789760790.686509:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27713536 -> 27717632 00002000:00010000:1.0:1789760790.686515:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760789 -> 1789760790 00002000:00010000:1.0:1789760790.686518:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760789 -> 1789760790 00010000:00010000:1.0:1789760790.686531:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545480ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27713536->27717631] (req 27713536->27717631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.686544:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545480ce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27713536->27717631] (req 27713536->27717631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.686553:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545480ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27713536->27717631] (req 27713536->27717631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.686560:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760790.686565:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545480ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27713536->27717631] (req 27713536->27717631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.686574:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545480ce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27713536->27717631] (req 27713536->27717631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.686595:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545480ce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27713536->27717631] (req 27713536->27717631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.686603:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x49367483545480ce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27713536->27717631] (req 27713536->27717631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.686868:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01375180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495950208:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51522us (52125us total) trans 4294974083 rc 0/0 00000100:00100000:1.0:1789760790.686878:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6795 00000100:00100000:1.0:1789760790.686888:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774915590611 time 774956047182 state 0x2 ref 0 00000100:00100000:2.0:1789760790.693927:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760790.694084:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495950336 00000100:00100000:0.0:1789760790.694095:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495950336 00000100:00100000:0.0:1789760790.694152:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774915590611 time 774956047182 state 0x0 ref 1 00000100:00100000:0.0:1789760790.694168:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774956307598 time 774956047182 state 0x1 ref 1 00000100:00100000:0.0:1789760790.694311:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774956307598 time 775015026816 state 0x0 ref 1 00000100:00100000:0.0:1789760790.694327:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495950336:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760790.694372:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27717632, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:0.0:1789760790.694382:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545480d5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.694399:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760790.694403:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.694410:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545480d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27717632->27721727] (req 27717632->27721727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.694424:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545480d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27717632->27721727] (req 27717632->27721727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760790.719487:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27717632 -> 27721728 00010000:00010000:0.0:1789760790.719509:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545480d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27717632->27721727] (req 27717632->27721727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.719523:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545480d5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27717632->27721727] (req 27717632->27721727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.719533:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545480d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27717632->27721727] (req 27717632->27721727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.719540:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760790.719545:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545480d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27717632->27721727] (req 27717632->27721727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.719554:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545480d5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27717632->27721727] (req 27717632->27721727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.719574:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545480d5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27717632->27721727] (req 27717632->27721727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.719582:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545480d5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27717632->27721727] (req 27717632->27721727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.719682:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495950336:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25344us (25750us total) trans 4294974084 rc 0/0 00000100:00100000:0.0:1789760790.719698:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6796 00000100:00100000:0.0:1789760790.719708:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774956307598 time 775015026816 state 0x2 ref 0 00000001:00080000:0.0:1789760790.722400:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974084 is committed 00000100:00100000:3.0:1789760790.723867:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760790.723979:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495950464 00000100:00100000:0.0:1789760790.723990:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495950464 00000100:00100000:0.0:1789760790.724159:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 774956307598 time 775015026816 state 0x0 ref 1 00000100:00100000:0.0:1789760790.724175:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775015287232 time 775015026816 state 0x1 ref 1 00000100:00100000:0.0:1789760790.724206:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775015287232 time 775044922876 state 0x0 ref 1 00000100:00100000:0.0:1789760790.724219:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495950464:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760790.724266:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27721728, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:0.0:1789760790.724276:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545480dc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.724317:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760790.724322:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.724329:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545480dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27721728->27725823] (req 27721728->27725823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.724343:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545480dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27721728->27725823] (req 27721728->27725823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.756374:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974085 is committed 00002000:00010000:2.0:1789760790.756480:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27721728 -> 27725824 00010000:00010000:2.0:1789760790.756496:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545480dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27721728->27725823] (req 27721728->27725823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.756510:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545480dc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27721728->27725823] (req 27721728->27725823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.756520:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545480dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27721728->27725823] (req 27721728->27725823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.756539:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760790.756545:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545480dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27721728->27725823] (req 27721728->27725823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.756554:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545480dc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27721728->27725823] (req 27721728->27725823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.756573:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545480dc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27721728->27725823] (req 27721728->27725823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.756582:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545480dc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27721728->27725823] (req 27721728->27725823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.756689:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495950464:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32468us (32827us total) trans 4294974085 rc 0/0 00000100:00100000:2.0:1789760790.756700:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6797 00000100:00100000:2.0:1789760790.756710:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775015287232 time 775044922876 state 0x2 ref 0 00000100:00100000:3.0:1789760790.761223:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760790.762056:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495950592 00000100:00100000:0.0:1789760790.762069:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495950592 00000100:00100000:0.0:1789760790.762132:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775015287232 time 775044922876 state 0x0 ref 1 00000100:00100000:0.0:1789760790.762149:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775045183292 time 775044922876 state 0x1 ref 1 00000100:00100000:0.0:1789760790.762184:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775045183292 time 775082900221 state 0x0 ref 1 00000100:00100000:0.0:1789760790.762198:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495950592:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760790.762249:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27725824, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:0.0:1789760790.762258:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545480e3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.762274:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760790.762315:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.762322:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545480e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27725824->27729919] (req 27725824->27729919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.762336:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545480e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27725824->27729919] (req 27725824->27729919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.809602:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974086 is committed 00002000:00010000:2.0:1789760790.809810:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27725824 -> 27729920 00010000:00010000:2.0:1789760790.809826:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545480e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27725824->27729919] (req 27725824->27729919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.809839:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545480e3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27725824->27729919] (req 27725824->27729919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.809850:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545480e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27725824->27729919] (req 27725824->27729919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.809863:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760790.809868:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545480e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27725824->27729919] (req 27725824->27729919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.809878:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545480e3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27725824->27729919] (req 27725824->27729919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.809960:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545480e3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27725824->27729919] (req 27725824->27729919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.809969:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545480e3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27725824->27729919] (req 27725824->27729919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.810167:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495950592:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47968us (48953us total) trans 4294974086 rc 0/0 00000100:00100000:2.0:1789760790.810178:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6798 00000100:00100000:2.0:1789760790.810188:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775045183292 time 775082900221 state 0x2 ref 0 00000100:00100000:3.0:1789760790.813847:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760790.813927:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495950720 00000100:00100000:0.0:1789760790.813937:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495950720 00000100:00100000:0.0:1789760790.814004:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775045183292 time 775082900221 state 0x0 ref 1 00000100:00100000:0.0:1789760790.814021:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775083160637 time 775082900221 state 0x1 ref 1 00000100:00100000:0.0:1789760790.814056:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775083160637 time 775134772974 state 0x0 ref 1 00000100:00100000:0.0:1789760790.814073:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495950720:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760790.814119:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27729920, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:0.0:1789760790.814129:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545480ea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.814146:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760790.814150:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.814157:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545480ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27729920->27734015] (req 27729920->27734015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.814171:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545480ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27729920->27734015] (req 27729920->27734015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760790.844480:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27729920 -> 27734016 00010000:00010000:2.0:1789760790.844499:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545480ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27729920->27734015] (req 27729920->27734015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.844512:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545480ea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27729920->27734015] (req 27729920->27734015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.844525:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545480ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27729920->27734015] (req 27729920->27734015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.844532:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760790.844536:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545480ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27729920->27734015] (req 27729920->27734015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.844546:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545480ea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27729920->27734015] (req 27729920->27734015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.844563:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545480ea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27729920->27734015] (req 27729920->27734015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.844571:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545480ea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27729920->27734015] (req 27729920->27734015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.844669:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495950720:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30595us (30828us total) trans 4294974087 rc 0/0 00000100:00100000:2.0:1789760790.844688:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6799 00000100:00100000:2.0:1789760790.844697:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775083160637 time 775134772974 state 0x2 ref 0 00000001:00080000:3.0:1789760790.848472:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974087 is committed 00000100:00100000:2.0:1789760790.848621:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760790.848708:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495950848 00000100:00100000:0.0:1789760790.848718:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495950848 00000100:00100000:0.0:1789760790.848773:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775083160637 time 775134772974 state 0x0 ref 1 00000100:00100000:0.0:1789760790.848788:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775135033390 time 775134772974 state 0x1 ref 1 00000100:00100000:0.0:1789760790.848816:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775135033390 time 775169532602 state 0x0 ref 1 00000100:00100000:0.0:1789760790.848828:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495950848:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760790.848873:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27734016, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:0.0:1789760790.848881:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545480f1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.848911:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760790.848916:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.848923:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545480f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27734016->27738111] (req 27734016->27738111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.848935:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545480f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27734016->27738111] (req 27734016->27738111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.902380:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974088 is committed 00002000:00010000:0.0:1789760790.902477:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27734016 -> 27738112 00010000:00010000:0.0:1789760790.902493:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545480f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27734016->27738111] (req 27734016->27738111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.902506:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545480f1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27734016->27738111] (req 27734016->27738111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.902515:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545480f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27734016->27738111] (req 27734016->27738111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.902524:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760790.902528:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545480f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27734016->27738111] (req 27734016->27738111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.902537:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545480f1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27734016->27738111] (req 27734016->27738111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.902557:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545480f1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27734016->27738111] (req 27734016->27738111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.902564:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x49367483545480f1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27734016->27738111] (req 27734016->27738111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.902664:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495950848:12345-192.168.203.9@tcp:4:dd.0 Request processed in 53825us (54035us total) trans 4294974088 rc 0/0 00000100:00100000:0.0:1789760790.902683:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6800 00000100:00100000:0.0:1789760790.902693:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775135033390 time 775169532602 state 0x2 ref 0 00000100:00100000:0.0:1789760790.905909:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760790.905988:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495950976 00000100:00100000:2.0:1789760790.906008:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495950976 00000100:00100000:2.0:1789760790.906059:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775135033390 time 775169532602 state 0x0 ref 1 00000100:00100000:2.0:1789760790.906074:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775169793018 time 775169532602 state 0x1 ref 1 00000100:00100000:2.0:1789760790.906151:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775169793018 time 775226867604 state 0x0 ref 1 00000100:00100000:2.0:1789760790.906164:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01376300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495950976:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760790.906207:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27738112, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:2.0:1789760790.906216:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545480f8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.906232:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760790.906237:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.906244:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545480f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27738112->27742207] (req 27738112->27742207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.906257:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545480f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27738112->27742207] (req 27738112->27742207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760790.955400:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27738112 -> 27742208 00010000:00010000:3.0:1789760790.955419:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545480f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27738112->27742207] (req 27738112->27742207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.955432:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545480f8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27738112->27742207] (req 27738112->27742207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.955442:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545480f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27738112->27742207] (req 27738112->27742207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.955450:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760790.955455:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545480f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27738112->27742207] (req 27738112->27742207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.955465:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545480f8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27738112->27742207] (req 27738112->27742207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.955496:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545480f8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27738112->27742207] (req 27738112->27742207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.955505:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545480f8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27738112->27742207] (req 27738112->27742207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.955605:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01376300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495950976:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49437us (49702us total) trans 4294974089 rc 0/0 00000100:00100000:3.0:1789760790.955616:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6801 00000100:00100000:3.0:1789760790.955627:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775169793018 time 775226867604 state 0x2 ref 0 00000001:00080000:1.0:1789760790.956754:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974089 is committed 00000100:00100000:3.0:1789760790.959765:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760790.959863:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495951104 00000100:00100000:2.0:1789760790.959874:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495951104 00000100:00100000:2.0:1789760790.959925:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775169793018 time 775226867604 state 0x0 ref 1 00000100:00100000:2.0:1789760790.960058:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775227128020 time 775226867604 state 0x1 ref 1 00000100:00100000:2.0:1789760790.960119:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775227128020 time 775280835019 state 0x0 ref 1 00000100:00100000:2.0:1789760790.960132:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495951104:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760790.960181:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27742208, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:2.0:1789760790.960191:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545480ff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.960208:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760790.960213:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.960219:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545480ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27742208->27746303] (req 27742208->27746303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.960233:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545480ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27742208->27746303] (req 27742208->27746303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.993856:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974090 is committed 00002000:00010000:2.0:1789760790.993959:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27742208 -> 27746304 00010000:00010000:2.0:1789760790.993975:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545480ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27742208->27746303] (req 27742208->27746303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.993987:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545480ff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27742208->27746303] (req 27742208->27746303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.994070:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545480ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27742208->27746303] (req 27742208->27746303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.994079:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760790.994084:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545480ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27742208->27746303] (req 27742208->27746303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.994094:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545480ff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27742208->27746303] (req 27742208->27746303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.994115:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545480ff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27742208->27746303] (req 27742208->27746303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.994123:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545480ff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27742208->27746303] (req 27742208->27746303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.994211:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495951104:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34077us (34557us total) trans 4294974090 rc 0/0 00000100:00100000:2.0:1789760790.994222:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6802 00000100:00100000:2.0:1789760790.994231:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775227128020 time 775280835019 state 0x2 ref 0 00000100:00100000:3.0:1789760790.998368:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760790.998495:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495951232 00000100:00100000:0.0:1789760790.998504:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495951232 00000100:00100000:0.0:1789760790.998553:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775227128020 time 775280835019 state 0x0 ref 1 00000100:00100000:0.0:1789760790.998568:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775281095435 time 775280835019 state 0x1 ref 1 00000100:00100000:0.0:1789760790.998598:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775281095435 time 775319314465 state 0x0 ref 1 00000100:00100000:0.0:1789760790.998610:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034ce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495951232:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760790.998658:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27746304, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:0.0:1789760790.998667:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354548106 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.998693:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760790.998698:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.998704:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354548106 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27746304->27750399] (req 27746304->27750399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.998719:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354548106 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27746304->27750399] (req 27746304->27750399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.029182:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974091 is committed 00002000:00010000:0.0:1789760791.029335:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27746304 -> 27750400 00010000:00010000:0.0:1789760791.029351:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354548106 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27746304->27750399] (req 27746304->27750399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.029364:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354548106 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27746304->27750399] (req 27746304->27750399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.029374:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354548106 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27746304->27750399] (req 27746304->27750399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.029383:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ae00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760791.029388:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354548106 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27746304->27750399] (req 27746304->27750399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.029397:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354548106 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27746304->27750399] (req 27746304->27750399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.029416:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354548106 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27746304->27750399] (req 27746304->27750399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.029424:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354548106 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27746304->27750399] (req 27746304->27750399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.029554:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034ce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495951232:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30941us (31191us total) trans 4294974091 rc 0/0 00000100:00100000:0.0:1789760791.029565:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6803 00000100:00100000:0.0:1789760791.029574:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775281095435 time 775319314465 state 0x2 ref 0 00000100:00100000:0.0:1789760791.034127:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760791.034196:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495951360 00000100:00100000:3.0:1789760791.034205:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495951360 00000100:00100000:3.0:1789760791.034393:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775281095435 time 775319314465 state 0x0 ref 1 00000100:00100000:3.0:1789760791.034407:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775319574881 time 775319314465 state 0x1 ref 1 00000100:00100000:3.0:1789760791.034435:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775319574881 time 775355151668 state 0x0 ref 1 00000100:00100000:3.0:1789760791.034448:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01374e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495951360:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760791.034495:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27750400, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:3.0:1789760791.034504:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454810d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.034520:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760791.034526:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.034532:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454810d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27750400->27754495] (req 27750400->27754495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.034551:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454810d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27750400->27754495] (req 27750400->27754495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760791.038191:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789760791.070254:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27750400 -> 27754496 00010000:00010000:3.0:1789760791.070272:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454810d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27750400->27754495] (req 27750400->27754495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.070309:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454810d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27750400->27754495] (req 27750400->27754495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.070320:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454810d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27750400->27754495] (req 27750400->27754495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.070328:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760791.070334:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454810d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27750400->27754495] (req 27750400->27754495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.070343:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454810d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27750400->27754495] (req 27750400->27754495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.070363:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454810d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27750400->27754495] (req 27750400->27754495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.070371:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454810d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27750400->27754495] (req 27750400->27754495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.070942:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01374e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495951360:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36490us (36817us total) trans 4294974092 rc 0/0 00000100:00100000:3.0:1789760791.070953:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6804 00000100:00100000:3.0:1789760791.070963:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775319574881 time 775355151668 state 0x2 ref 0 00000001:00080000:0.0:1789760791.072577:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974092 is committed 00000100:00100000:0.0:1789760791.075312:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760791.075797:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495951488 00000100:00100000:2.0:1789760791.075806:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495951488 00000100:00100000:2.0:1789760791.075857:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775319574881 time 775355151668 state 0x0 ref 1 00000100:00100000:2.0:1789760791.075872:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775355412084 time 775355151668 state 0x1 ref 1 00000100:00100000:2.0:1789760791.075905:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775355412084 time 775396622070 state 0x0 ref 1 00000100:00100000:2.0:1789760791.075918:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01377800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495951488:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760791.075961:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27754496, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:2.0:1789760791.075971:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548114 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.075998:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760791.076003:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.076009:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548114 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27754496->27758591] (req 27754496->27758591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.076021:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548114 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27754496->27758591] (req 27754496->27758591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760791.117955:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27754496 -> 27758592 00010000:00010000:2.0:1789760791.117973:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548114 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27754496->27758591] (req 27754496->27758591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.117986:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548114 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27754496->27758591] (req 27754496->27758591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.118007:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548114 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27754496->27758591] (req 27754496->27758591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.118015:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760791.118020:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548114 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27754496->27758591] (req 27754496->27758591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.118030:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548114 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27754496->27758591] (req 27754496->27758591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.118050:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548114 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27754496->27758591] (req 27754496->27758591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.118060:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548114 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27754496->27758591] (req 27754496->27758591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.118151:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01377800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495951488:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42231us (42873us total) trans 4294974093 rc 0/0 00000100:00100000:2.0:1789760791.118162:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6805 00000100:00100000:2.0:1789760791.118172:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775355412084 time 775396622070 state 0x2 ref 0 00000001:00080000:0.0:1789760791.120717:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974093 is committed 00000100:00100000:2.0:1789760791.121490:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760791.121596:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495951616 00000100:00100000:0.0:1789760791.121604:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495951616 00000100:00100000:0.0:1789760791.121719:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775355412084 time 775396622070 state 0x0 ref 1 00000100:00100000:0.0:1789760791.121733:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775396882486 time 775396622070 state 0x1 ref 1 00000100:00100000:0.0:1789760791.121807:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775396882486 time 775442522176 state 0x0 ref 1 00000100:00100000:0.0:1789760791.121822:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fdca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495951616:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760791.121869:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27758592, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:0.0:1789760791.121878:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454811b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.121893:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760791.121897:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.121903:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454811b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27758592->27762687] (req 27758592->27762687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.121917:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454811b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27758592->27762687] (req 27758592->27762687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.174905:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974094 is committed 00002000:00010000:0.0:1789760791.174989:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27758592 -> 27762688 00010000:00010000:0.0:1789760791.175069:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454811b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27758592->27762687] (req 27758592->27762687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.175083:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454811b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27758592->27762687] (req 27758592->27762687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.175093:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454811b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27758592->27762687] (req 27758592->27762687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.175101:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ae00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760791.175106:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454811b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27758592->27762687] (req 27758592->27762687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.175116:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454811b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27758592->27762687] (req 27758592->27762687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.175136:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454811b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27758592->27762687] (req 27758592->27762687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.175144:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x493674835454811b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27758592->27762687] (req 27758592->27762687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.175416:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fdca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495951616:12345-192.168.203.9@tcp:4:dd.0 Request processed in 53593us (53927us total) trans 4294974094 rc 0/0 00000100:00100000:0.0:1789760791.175427:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6806 00000100:00100000:0.0:1789760791.175437:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775396882486 time 775442522176 state 0x2 ref 0 00000100:00100000:3.0:1789760791.179486:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760791.179715:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495951744 00000100:00100000:2.0:1789760791.179725:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495951744 00000100:00100000:2.0:1789760791.179777:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775396882486 time 775442522176 state 0x0 ref 1 00000100:00100000:2.0:1789760791.179792:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775442782592 time 775442522176 state 0x1 ref 1 00000100:00100000:2.0:1789760791.179822:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775442782592 time 775500539378 state 0x0 ref 1 00000100:00100000:2.0:1789760791.179835:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495951744:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760791.179881:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27762688, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:2.0:1789760791.179890:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548122 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.179907:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760791.179911:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.179918:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548122 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27762688->27766783] (req 27762688->27766783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.179932:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548122 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27762688->27766783] (req 27762688->27766783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.214953:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974095 is committed 00002000:00010000:2.0:1789760791.215126:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27762688 -> 27766784 00010000:00010000:2.0:1789760791.215142:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548122 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27762688->27766783] (req 27762688->27766783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.215156:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548122 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27762688->27766783] (req 27762688->27766783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.215165:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548122 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27762688->27766783] (req 27762688->27766783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.215173:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760791.215177:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548122 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27762688->27766783] (req 27762688->27766783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.215187:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548122 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27762688->27766783] (req 27762688->27766783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.215206:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548122 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27762688->27766783] (req 27762688->27766783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.215215:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548122 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27762688->27766783] (req 27762688->27766783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.215378:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495951744:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35540us (35895us total) trans 4294974095 rc 0/0 00000100:00100000:2.0:1789760791.215388:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6807 00000100:00100000:2.0:1789760791.215398:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775442782592 time 775500539378 state 0x2 ref 0 00000100:00100000:0.0:1789760791.219541:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760791.219638:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495951872 00000100:00100000:2.0:1789760791.219708:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495951872 00000100:00100000:2.0:1789760791.219764:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775442782592 time 775500539378 state 0x0 ref 1 00000100:00100000:2.0:1789760791.219779:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775500799794 time 775500539378 state 0x1 ref 1 00000100:00100000:2.0:1789760791.219844:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775500799794 time 775540560420 state 0x0 ref 1 00000100:00100000:2.0:1789760791.219856:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fba3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495951872:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760791.219901:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27766784, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:2.0:1789760791.219910:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548129 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.219926:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760791.219931:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.219937:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548129 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27766784->27770879] (req 27766784->27770879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.219950:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548129 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27766784->27770879] (req 27766784->27770879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.248408:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974096 is committed 00002000:00010000:0.0:1789760791.248436:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27766784 -> 27770880 00010000:00010000:0.0:1789760791.248452:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548129 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27766784->27770879] (req 27766784->27770879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.248465:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548129 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27766784->27770879] (req 27766784->27770879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.248475:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548129 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27766784->27770879] (req 27766784->27770879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.248482:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760791.248487:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548129 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27766784->27770879] (req 27766784->27770879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.248497:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548129 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27766784->27770879] (req 27766784->27770879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.248516:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548129 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27766784->27770879] (req 27766784->27770879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.248525:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548129 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27766784->27770879] (req 27766784->27770879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.248609:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fba3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495951872:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28750us (29073us total) trans 4294974096 rc 0/0 00000100:00100000:0.0:1789760791.248618:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6808 00000100:00100000:0.0:1789760791.248628:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775500799794 time 775540560420 state 0x2 ref 0 00000100:00100000:0.0:1789760791.253145:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760791.253251:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495952000 00000100:00100000:2.0:1789760791.253260:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495952000 00000100:00100000:2.0:1789760791.253399:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775500799794 time 775540560420 state 0x0 ref 1 00000100:00100000:2.0:1789760791.253415:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775540820836 time 775540560420 state 0x1 ref 1 00000100:00100000:2.0:1789760791.253442:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775540820836 time 775574158801 state 0x0 ref 1 00000100:00100000:2.0:1789760791.253454:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01374a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495952000:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760791.253499:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27770880, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:2.0:1789760791.253507:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548130 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.253630:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760791.253635:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.253641:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548130 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27770880->27774975] (req 27770880->27774975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.253726:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548130 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27770880->27774975] (req 27770880->27774975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.287771:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974097 is committed 00002000:00010000:2.0:1789760791.287894:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27770880 -> 27774976 00010000:00010000:2.0:1789760791.287909:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548130 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27770880->27774975] (req 27770880->27774975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.287929:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548130 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27770880->27774975] (req 27770880->27774975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.287937:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548130 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27770880->27774975] (req 27770880->27774975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.287944:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760791.287949:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548130 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27770880->27774975] (req 27770880->27774975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.287959:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548130 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27770880->27774975] (req 27770880->27774975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.287978:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548130 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27770880->27774975] (req 27770880->27774975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.287985:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548130 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27770880->27774975] (req 27770880->27774975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.288186:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01374a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495952000:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34730us (35045us total) trans 4294974097 rc 0/0 00000100:00100000:2.0:1789760791.288197:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6809 00000100:00100000:2.0:1789760791.288207:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775540820836 time 775574158801 state 0x2 ref 0 00000100:00100000:0.0:1789760791.292920:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760791.293073:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495952128 00000100:00100000:2.0:1789760791.293083:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495952128 00000100:00100000:2.0:1789760791.293136:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775540820836 time 775574158801 state 0x0 ref 1 00000100:00100000:2.0:1789760791.293152:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775574419217 time 775574158801 state 0x1 ref 1 00000100:00100000:2.0:1789760791.293211:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775574419217 time 775613927271 state 0x0 ref 1 00000100:00100000:2.0:1789760791.293224:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01375180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495952128:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760791.293267:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27774976, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:2.0:1789760791.293275:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548137 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.293318:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760791.293330:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.293336:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548137 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27774976->27779071] (req 27774976->27779071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.293350:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548137 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27774976->27779071] (req 27774976->27779071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.324329:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974098 is committed 00002000:00010000:2.0:1789760791.324808:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27774976 -> 27779072 00010000:00010000:2.0:1789760791.324823:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548137 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27774976->27779071] (req 27774976->27779071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.324835:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548137 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27774976->27779071] (req 27774976->27779071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.324849:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548137 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27774976->27779071] (req 27774976->27779071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.324857:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760791.324861:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548137 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27774976->27779071] (req 27774976->27779071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.324871:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548137 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27774976->27779071] (req 27774976->27779071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.324901:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548137 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27774976->27779071] (req 27774976->27779071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.324910:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548137 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27774976->27779071] (req 27774976->27779071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.328212:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01375180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495952128:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34980us (35293us total) trans 4294974098 rc 0/0 00000100:00100000:2.0:1789760791.328228:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6810 00000100:00100000:2.0:1789760791.328240:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775574419217 time 775613927271 state 0x2 ref 0 00000100:00100000:0.0:1789760791.328823:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760791.328945:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495952256 00000100:00100000:2.0:1789760791.328955:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495952256 00000100:00100000:2.0:1789760791.329058:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775574419217 time 775613927271 state 0x0 ref 1 00000100:00100000:2.0:1789760791.329072:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775614187687 time 775613927271 state 0x1 ref 1 00000100:00100000:2.0:1789760791.329157:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775614187687 time 775649816073 state 0x0 ref 1 00000100:00100000:2.0:1789760791.329170:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01376300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495952256:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760791.329213:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27779072, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:2.0:1789760791.329221:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454813e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.329236:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760791.329240:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.329246:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454813e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27779072->27783167] (req 27779072->27783167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.329259:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454813e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27779072->27783167] (req 27779072->27783167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.366378:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974099 is committed 00002000:00010000:2.0:1789760791.366631:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27779072 -> 27783168 00010000:00010000:2.0:1789760791.366646:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454813e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27779072->27783167] (req 27779072->27783167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.366724:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454813e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27779072->27783167] (req 27779072->27783167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.366735:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454813e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27779072->27783167] (req 27779072->27783167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.366742:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760791.366747:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454813e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27779072->27783167] (req 27779072->27783167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.366757:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454813e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27779072->27783167] (req 27779072->27783167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.366777:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454813e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27779072->27783167] (req 27779072->27783167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.366785:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x493674835454813e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27779072->27783167] (req 27779072->27783167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.366870:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01376300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495952256:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37697us (38044us total) trans 4294974099 rc 0/0 00000100:00100000:2.0:1789760791.366879:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6811 00000100:00100000:2.0:1789760791.366889:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775614187687 time 775649816073 state 0x2 ref 0 00000100:00100000:0.0:1789760791.372988:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760791.373067:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495952384 00000100:00100000:2.0:1789760791.373075:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495952384 00000100:00100000:2.0:1789760791.373122:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775614187687 time 775649816073 state 0x0 ref 1 00000100:00100000:2.0:1789760791.373134:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775650076489 time 775649816073 state 0x1 ref 1 00000100:00100000:2.0:1789760791.373158:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775650076489 time 775693876466 state 0x0 ref 1 00000100:00100000:2.0:1789760791.373167:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01374e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495952384:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760791.373206:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27783168, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:2.0:1789760791.373213:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548145 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.373226:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760791.373229:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.373242:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548145 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27783168->27787263] (req 27783168->27787263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.373253:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548145 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27783168->27787263] (req 27783168->27787263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.396143:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760791.416606:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27783168 -> 27787264 00010000:00010000:2.0:1789760791.416627:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548145 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27783168->27787263] (req 27783168->27787263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.416641:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548145 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27783168->27787263] (req 27783168->27787263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.416740:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974100 is committed 00010000:00010000:2.0:1789760791.416856:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548145 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27783168->27787263] (req 27783168->27787263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.416865:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760791.416870:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548145 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27783168->27787263] (req 27783168->27787263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.416880:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548145 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27783168->27787263] (req 27783168->27787263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.416907:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548145 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27783168->27787263] (req 27783168->27787263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.416915:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548145 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27783168->27787263] (req 27783168->27787263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.417101:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01374e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495952384:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43929us (44120us total) trans 4294974100 rc 0/0 00000100:00100000:2.0:1789760791.417112:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6812 00000100:00100000:2.0:1789760791.417122:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775650076489 time 775693876466 state 0x2 ref 0 00000100:00100000:0.0:1789760791.427448:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760791.428840:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495952512 00000100:00100000:2.0:1789760791.428853:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495952512 00000100:00100000:2.0:1789760791.428908:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775650076489 time 775693876466 state 0x0 ref 1 00000100:00100000:2.0:1789760791.428923:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775694136882 time 775693876466 state 0x1 ref 1 00000100:00100000:2.0:1789760791.429025:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775694136882 time 775749740580 state 0x0 ref 1 00000100:00100000:2.0:1789760791.429037:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01377800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495952512:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760791.429081:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27787264, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:2.0:1789760791.429089:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454814c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.429105:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760791.429110:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.429116:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454814c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27787264->27791359] (req 27787264->27791359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.429128:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454814c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27787264->27791359] (req 27787264->27791359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.475362:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974101 is committed 00002000:00010000:2.0:1789760791.475514:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27787264 -> 27791360 00010000:00010000:2.0:1789760791.475529:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454814c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27787264->27791359] (req 27787264->27791359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.475541:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454814c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27787264->27791359] (req 27787264->27791359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.475550:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454814c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27787264->27791359] (req 27787264->27791359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.475558:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760791.475563:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454814c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27787264->27791359] (req 27787264->27791359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.475572:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454814c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27787264->27791359] (req 27787264->27791359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.475594:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454814c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27787264->27791359] (req 27787264->27791359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.475607:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x493674835454814c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27787264->27791359] (req 27787264->27791359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.475814:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01377800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495952512:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46774us (48370us total) trans 4294974101 rc 0/0 00000100:00100000:2.0:1789760791.475825:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6813 00000100:00100000:2.0:1789760791.475836:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775694136882 time 775749740580 state 0x2 ref 0 00000100:00100000:0.0:1789760791.482410:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760791.482517:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495952640 00000100:00100000:2.0:1789760791.482526:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495952640 00000100:00100000:2.0:1789760791.482583:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775694136882 time 775749740580 state 0x0 ref 1 00000100:00100000:2.0:1789760791.482598:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775750000996 time 775749740580 state 0x1 ref 1 00000100:00100000:2.0:1789760791.482681:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775750000996 time 775803397264 state 0x0 ref 1 00000100:00100000:2.0:1789760791.482695:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01376a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495952640:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760791.482746:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27791360, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:2.0:1789760791.482755:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548153 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.482770:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760791.482775:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.482781:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548153 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27791360->27795455] (req 27791360->27795455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.482799:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548153 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27791360->27795455] (req 27791360->27795455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760791.532573:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27791360 -> 27795456 00010000:00010000:2.0:1789760791.532594:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548153 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27791360->27795455] (req 27791360->27795455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.532599:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974102 is committed 00010000:00010000:2.0:1789760791.532612:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548153 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27791360->27795455] (req 27791360->27795455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.532622:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548153 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27791360->27795455] (req 27791360->27795455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.532630:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760791.532635:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548153 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27791360->27795455] (req 27791360->27795455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.532644:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548153 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27791360->27795455] (req 27791360->27795455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.532664:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548153 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27791360->27795455] (req 27791360->27795455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.532805:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548153 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27791360->27795455] (req 27791360->27795455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.532902:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01376a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495952640:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50206us (50497us total) trans 4294974102 rc 0/0 00000100:00100000:2.0:1789760791.532913:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6814 00000100:00100000:2.0:1789760791.532924:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775750000996 time 775803397264 state 0x2 ref 0 00000100:00100000:0.0:1789760791.536565:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760791.536714:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495952768 00000100:00100000:3.0:1789760791.536724:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495952768 00000100:00100000:3.0:1789760791.536778:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775750000996 time 775803397264 state 0x0 ref 1 00000100:00100000:3.0:1789760791.536793:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775803657680 time 775803397264 state 0x1 ref 1 00000100:00100000:3.0:1789760791.536827:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775803657680 time 775857544026 state 0x0 ref 1 00000100:00100000:3.0:1789760791.536840:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01377100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495952768:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760791.536960:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27795456, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:0.0:1789760791.536970:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454815a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.536998:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760791.537003:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.537009:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454815a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27795456->27799551] (req 27795456->27799551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.537024:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454815a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27795456->27799551] (req 27795456->27799551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.566087:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974103 is committed 00002000:00010000:3.0:1789760791.566337:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27795456 -> 27799552 00010000:00010000:3.0:1789760791.566353:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454815a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27795456->27799551] (req 27795456->27799551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.566365:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454815a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27795456->27799551] (req 27795456->27799551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.566379:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454815a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27795456->27799551] (req 27795456->27799551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.566387:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760791.566392:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454815a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27795456->27799551] (req 27795456->27799551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.566401:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454815a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27795456->27799551] (req 27795456->27799551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.566421:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454815a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27795456->27799551] (req 27795456->27799551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.566428:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454815a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27795456->27799551] (req 27795456->27799551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.566521:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01377100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495952768:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29679us (29959us total) trans 4294974103 rc 0/0 00000100:00100000:3.0:1789760791.566531:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6815 00000100:00100000:3.0:1789760791.566541:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775803657680 time 775857544026 state 0x2 ref 0 00000100:00100000:3.0:1789760791.571719:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760791.572365:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495952896 00000100:00100000:2.0:1789760791.572375:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495952896 00000100:00100000:2.0:1789760791.572427:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775803657680 time 775857544026 state 0x0 ref 1 00000100:00100000:2.0:1789760791.572444:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775857804442 time 775857544026 state 0x1 ref 1 00000100:00100000:2.0:1789760791.572468:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775857804442 time 775893184319 state 0x0 ref 1 00000100:00100000:2.0:1789760791.572479:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495952896:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760791.572525:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27799552, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:2.0:1789760791.572534:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548161 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.572550:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760791.572554:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.572560:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548161 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27799552->27803647] (req 27799552->27803647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.572573:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548161 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27799552->27803647] (req 27799552->27803647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.601396:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974104 is committed 00002000:00010000:3.0:1789760791.601506:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27799552 -> 27803648 00010000:00010000:3.0:1789760791.601521:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548161 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27799552->27803647] (req 27799552->27803647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.601540:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548161 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27799552->27803647] (req 27799552->27803647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.601549:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548161 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27799552->27803647] (req 27799552->27803647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.601555:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760791.601559:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548161 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27799552->27803647] (req 27799552->27803647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.601568:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548161 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27799552->27803647] (req 27799552->27803647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.601589:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548161 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27799552->27803647] (req 27799552->27803647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.602146:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548161 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27799552->27803647] (req 27799552->27803647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.602254:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495952896:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29770us (30538us total) trans 4294974104 rc 0/0 00000100:00100000:3.0:1789760791.602265:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6816 00000100:00100000:3.0:1789760791.602275:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775857804442 time 775893184319 state 0x2 ref 0 00000100:00100000:0.0:1789760791.605762:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760791.605895:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495953024 00000100:00100000:2.0:1789760791.605905:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495953024 00000100:00100000:2.0:1789760791.605958:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775857804442 time 775893184319 state 0x0 ref 1 00000100:00100000:2.0:1789760791.605972:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775893444735 time 775893184319 state 0x1 ref 1 00000100:00100000:2.0:1789760791.606006:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775893444735 time 775926722878 state 0x0 ref 1 00000100:00100000:2.0:1789760791.606019:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01375880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495953024:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760791.606077:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27803648, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:2.0:1789760791.606086:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354548168 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.606103:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760791.606107:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.606114:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354548168 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27803648->27807743] (req 27803648->27807743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.606127:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354548168 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27803648->27807743] (req 27803648->27807743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760791.638992:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27803648 -> 27807744 00010000:00010000:2.0:1789760791.639016:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354548168 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27803648->27807743] (req 27803648->27807743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.639036:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354548168 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27803648->27807743] (req 27803648->27807743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.639046:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354548168 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27803648->27807743] (req 27803648->27807743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.639054:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760791.639070:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354548168 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27803648->27807743] (req 27803648->27807743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.639082:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354548168 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27803648->27807743] (req 27803648->27807743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.639105:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354548168 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27803648->27807743] (req 27803648->27807743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.639113:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354548168 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27803648->27807743] (req 27803648->27807743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.639220:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01375880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495953024:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33196us (33459us total) trans 4294974105 rc 0/0 00000100:00100000:2.0:1789760791.639233:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6817 00000100:00100000:2.0:1789760791.639245:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775893444735 time 775926722878 state 0x2 ref 0 00000001:00080000:1.0:1789760791.639436:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974105 is committed 00000100:00100000:2.0:1789760791.647241:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760791.649177:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495953152 00000100:00100000:3.0:1789760791.649192:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495953152 00000100:00100000:3.0:1789760791.649258:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775893444735 time 775926722878 state 0x0 ref 1 00000100:00100000:3.0:1789760791.649274:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775926983294 time 775926722878 state 0x1 ref 1 00000100:00100000:3.0:1789760791.649384:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775926983294 time 775970099541 state 0x0 ref 1 00000100:00100000:3.0:1789760791.649398:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fdd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495953152:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760791.649450:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27807744, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:3.0:1789760791.649458:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454816f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.649474:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760791.649479:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.649485:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454816f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27807744->27811839] (req 27807744->27811839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.649499:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454816f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27807744->27811839] (req 27807744->27811839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.704488:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974106 is committed 00002000:00010000:3.0:1789760791.704694:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27807744 -> 27811840 00002000:00010000:3.0:1789760791.704699:0:13059:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760790 -> 1789760791 00002000:00010000:3.0:1789760791.704702:0:13059:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760790 -> 1789760791 00010000:00010000:3.0:1789760791.704716:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454816f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27807744->27811839] (req 27807744->27811839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.704729:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454816f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27807744->27811839] (req 27807744->27811839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.704740:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454816f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27807744->27811839] (req 27807744->27811839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.704748:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760791.704752:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454816f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27807744->27811839] (req 27807744->27811839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.704762:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454816f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27807744->27811839] (req 27807744->27811839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.704784:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454816f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27807744->27811839] (req 27807744->27811839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.704792:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454816f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27807744->27811839] (req 27807744->27811839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.704884:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fdd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495953152:12345-192.168.203.9@tcp:4:dd.0 Request processed in 55483us (57648us total) trans 4294974106 rc 0/0 00000100:00100000:3.0:1789760791.704894:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6818 00000100:00100000:3.0:1789760791.704906:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775926983294 time 775970099541 state 0x2 ref 0 00000100:00100000:0.0:1789760791.717080:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760791.717242:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495953280 00000100:00100000:3.0:1789760791.717251:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495953280 00000100:00100000:3.0:1789760791.717397:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775926983294 time 775970099541 state 0x0 ref 1 00000100:00100000:3.0:1789760791.717414:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775970359957 time 775970099541 state 0x1 ref 1 00000100:00100000:3.0:1789760791.717496:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775970359957 time 776038211409 state 0x0 ref 1 00000100:00100000:3.0:1789760791.717510:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01377b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495953280:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760791.717568:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27811840, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:3.0:1789760791.717579:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548176 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.717596:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760791.717601:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.717620:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548176 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27811840->27815935] (req 27811840->27815935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.717634:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548176 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27811840->27815935] (req 27811840->27815935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760791.766630:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27811840 -> 27815936 00010000:00010000:3.0:1789760791.766729:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548176 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27811840->27815935] (req 27811840->27815935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.766742:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548176 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27811840->27815935] (req 27811840->27815935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.766751:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548176 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27811840->27815935] (req 27811840->27815935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.766758:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760791.766763:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548176 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27811840->27815935] (req 27811840->27815935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.766772:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548176 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27811840->27815935] (req 27811840->27815935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.766793:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548176 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27811840->27815935] (req 27811840->27815935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.766801:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548176 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27811840->27815935] (req 27811840->27815935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.766929:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01377b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495953280:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49417us (49856us total) trans 4294974107 rc 0/0 00000100:00100000:3.0:1789760791.766940:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6819 00000100:00100000:3.0:1789760791.766950:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775970359957 time 776038211409 state 0x2 ref 0 00000001:00080000:0.0:1789760791.767138:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974107 is committed 00000100:00100000:0.0:1789760791.770396:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760791.770487:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495953408 00000100:00100000:2.0:1789760791.770496:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495953408 00000100:00100000:2.0:1789760791.770550:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 775970359957 time 776038211409 state 0x0 ref 1 00000100:00100000:2.0:1789760791.770564:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776038471825 time 776038211409 state 0x1 ref 1 00000100:00100000:2.0:1789760791.770590:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776038471825 time 776091307782 state 0x0 ref 1 00000100:00100000:2.0:1789760791.770602:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01375f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495953408:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760791.770644:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27815936, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:2.0:1789760791.770671:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454817d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.770686:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760791.770691:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.770697:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454817d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27815936->27820031] (req 27815936->27820031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.770711:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454817d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27815936->27820031] (req 27815936->27820031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.804194:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974108 is committed 00002000:00010000:1.0:1789760791.804344:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27815936 -> 27820032 00010000:00010000:1.0:1789760791.804359:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454817d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27815936->27820031] (req 27815936->27820031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.804372:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454817d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27815936->27820031] (req 27815936->27820031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.804381:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454817d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27815936->27820031] (req 27815936->27820031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.804388:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760791.804393:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454817d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27815936->27820031] (req 27815936->27820031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.804401:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454817d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27815936->27820031] (req 27815936->27820031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.804426:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454817d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27815936->27820031] (req 27815936->27820031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.804434:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454817d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27815936->27820031] (req 27815936->27820031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.804523:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01375f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495953408:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33917us (34131us total) trans 4294974108 rc 0/0 00000100:00100000:1.0:1789760791.804534:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6820 00000100:00100000:1.0:1789760791.804543:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776038471825 time 776091307782 state 0x2 ref 0 00000100:00100000:2.0:1789760791.809418:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760791.809683:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495953536 00000100:00100000:3.0:1789760791.809693:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495953536 00000100:00100000:3.0:1789760791.809745:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776038471825 time 776091307782 state 0x0 ref 1 00000100:00100000:3.0:1789760791.809770:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776091568198 time 776091307782 state 0x1 ref 1 00000100:00100000:3.0:1789760791.809798:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776091568198 time 776130514773 state 0x0 ref 1 00000100:00100000:3.0:1789760791.809809:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495953536:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760791.809856:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27820032, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:3.0:1789760791.809865:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548184 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.809880:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760791.809884:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.809891:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548184 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27820032->27824127] (req 27820032->27824127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.809905:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548184 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27820032->27824127] (req 27820032->27824127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.838960:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f02769f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520317184:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760791.839100:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760791.839222:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520317184 00000100:00100000:1.0:1789760791.839239:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520317184 00000100:00100000:1.0:1789760791.839326:0:6711:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 219 00000100:00100000:1.0:1789760791.839336:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520317184:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760791.839358:0:6711:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528154624 pending 1703936 free 3830448128 avail 3802136576 00000100:00100000:1.0:1789760791.839405:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520317184:12345-0@lo:13:osp-pre-0-0.0 Request processed in 71us (305us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760791.839412:0:6711:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 219 00000100:00100000:1.0:1789760791.839450:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f02769f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520317184:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1789760791.839567:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789760791.851884:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27820032 -> 27824128 00010000:00010000:0.0:1789760791.851907:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548184 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27820032->27824127] (req 27820032->27824127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.851920:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548184 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27820032->27824127] (req 27820032->27824127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.851931:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548184 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27820032->27824127] (req 27820032->27824127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.851939:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760791.851945:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548184 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27820032->27824127] (req 27820032->27824127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.851954:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548184 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27820032->27824127] (req 27820032->27824127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.851982:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548184 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27820032->27824127] (req 27820032->27824127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.852003:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548184 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27820032->27824127] (req 27820032->27824127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.852103:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495953536:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42291us (42690us total) trans 4294974109 rc 0/0 00000100:00100000:0.0:1789760791.852114:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6821 00000100:00100000:0.0:1789760791.852124:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776091568198 time 776130514773 state 0x2 ref 0 00000001:00080000:1.0:1789760791.853072:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974109 is committed 00000100:00100000:0.0:1789760791.865486:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760791.865595:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495953664 00000100:00100000:3.0:1789760791.865604:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495953664 00000100:00100000:3.0:1789760791.865683:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776091568198 time 776130514773 state 0x0 ref 1 00000100:00100000:3.0:1789760791.865699:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776130775189 time 776130514773 state 0x1 ref 1 00000100:00100000:3.0:1789760791.865728:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776130775189 time 776186444463 state 0x0 ref 1 00000100:00100000:3.0:1789760791.865740:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01375500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495953664:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760791.865791:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27824128, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:3.0:1789760791.865800:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454818b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.865822:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760791.865827:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.865834:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454818b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27824128->27828223] (req 27824128->27828223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.865850:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454818b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27824128->27828223] (req 27824128->27828223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.894406:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974110 is committed 00002000:00010000:2.0:1789760791.894542:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27824128 -> 27828224 00010000:00010000:2.0:1789760791.894560:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454818b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27824128->27828223] (req 27824128->27828223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.894575:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454818b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27824128->27828223] (req 27824128->27828223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.894585:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454818b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27824128->27828223] (req 27824128->27828223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.894594:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760791.894600:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454818b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27824128->27828223] (req 27824128->27828223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.894610:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454818b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27824128->27828223] (req 27824128->27828223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.894632:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454818b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27824128->27828223] (req 27824128->27828223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.894639:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454818b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27824128->27828223] (req 27824128->27828223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.894755:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01375500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495953664:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29012us (29276us total) trans 4294974110 rc 0/0 00000100:00100000:2.0:1789760791.894766:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6822 00000100:00100000:2.0:1789760791.894776:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776130775189 time 776186444463 state 0x2 ref 0 00000100:00100000:2.0:1789760791.903623:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760791.903728:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495953792 00000100:00100000:1.0:1789760791.903737:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495953792 00000100:00100000:1.0:1789760791.903796:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776130775189 time 776186444463 state 0x0 ref 1 00000100:00100000:1.0:1789760791.903815:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776186704879 time 776186444463 state 0x1 ref 1 00000100:00100000:1.0:1789760791.903847:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776186704879 time 776224563862 state 0x0 ref 1 00000100:00100000:1.0:1789760791.903859:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495953792:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760791.903909:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27828224, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:1.0:1789760791.903919:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548192 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.903935:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760791.903940:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.903946:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548192 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27828224->27832319] (req 27828224->27832319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.903959:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548192 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27828224->27832319] (req 27828224->27832319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.939152:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974111 is committed 00002000:00010000:3.0:1789760791.939326:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27828224 -> 27832320 00010000:00010000:3.0:1789760791.939342:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548192 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27828224->27832319] (req 27828224->27832319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.939355:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548192 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27828224->27832319] (req 27828224->27832319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.939364:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548192 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27828224->27832319] (req 27828224->27832319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.939371:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760791.939376:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548192 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27828224->27832319] (req 27828224->27832319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.939386:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548192 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27828224->27832319] (req 27828224->27832319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.939412:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548192 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27828224->27832319] (req 27828224->27832319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.939419:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548192 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27828224->27832319] (req 27828224->27832319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.939816:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495953792:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35954us (36196us total) trans 4294974111 rc 0/0 00000100:00100000:3.0:1789760791.939827:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6823 00000100:00100000:3.0:1789760791.939838:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776186704879 time 776224563862 state 0x2 ref 0 00000100:00100000:3.0:1789760791.944971:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760791.945129:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495953920 00000100:00100000:0.0:1789760791.945140:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495953920 00000100:00100000:0.0:1789760791.945194:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776186704879 time 776224563862 state 0x0 ref 1 00000100:00100000:0.0:1789760791.945209:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776224824278 time 776224563862 state 0x1 ref 1 00000100:00100000:0.0:1789760791.945268:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776224824278 time 776265984762 state 0x0 ref 1 00000100:00100000:0.0:1789760791.945313:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495953920:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760791.945368:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27832320, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:0.0:1789760791.945377:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354548199 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.945396:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:0.0:1789760791.945400:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.945407:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354548199 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27832320->27836415] (req 27832320->27836415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.945420:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354548199 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27832320->27836415] (req 27832320->27836415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.971461:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974112 is committed 00002000:00010000:3.0:1789760791.971594:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27832320 -> 27836416 00010000:00010000:3.0:1789760791.971612:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354548199 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27832320->27836415] (req 27832320->27836415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.971633:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354548199 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27832320->27836415] (req 27832320->27836415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.971643:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354548199 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27832320->27836415] (req 27832320->27836415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.971650:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760791.971655:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354548199 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27832320->27836415] (req 27832320->27836415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.971679:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354548199 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27832320->27836415] (req 27832320->27836415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.971699:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354548199 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27832320->27836415] (req 27832320->27836415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.971706:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x4936748354548199 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27832320->27836415] (req 27832320->27836415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.971795:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034ce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495953920:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26511us (26829us total) trans 4294974112 rc 0/0 00000100:00100000:3.0:1789760791.971806:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6824 00000100:00100000:3.0:1789760791.971821:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776224824278 time 776265984762 state 0x2 ref 0 00000100:00100000:0.0:1789760791.980771:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760791.981686:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495954048 00000100:00100000:2.0:1789760791.981699:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495954048 00000100:00100000:2.0:1789760791.981848:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776224824278 time 776265984762 state 0x0 ref 1 00000100:00100000:2.0:1789760791.981863:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776266245178 time 776265984762 state 0x1 ref 1 00000100:00100000:2.0:1789760791.981958:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776266245178 time 776302673324 state 0x0 ref 1 00000100:00100000:2.0:1789760791.981973:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01377480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495954048:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760791.982022:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27836416, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:2.0:1789760791.982031:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545481a0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.982047:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760791.982053:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.982060:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545481a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27836416->27840511] (req 27836416->27840511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.982073:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545481a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27836416->27840511] (req 27836416->27840511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.022323:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974113 is committed 00002000:00010000:3.0:1789760792.022659:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27836416 -> 27840512 00010000:00010000:3.0:1789760792.022686:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545481a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27836416->27840511] (req 27836416->27840511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.022699:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545481a0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27836416->27840511] (req 27836416->27840511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.022708:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545481a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27836416->27840511] (req 27836416->27840511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.022715:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760792.022721:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545481a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27836416->27840511] (req 27836416->27840511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.022730:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545481a0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27836416->27840511] (req 27836416->27840511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.022749:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545481a0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27836416->27840511] (req 27836416->27840511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.022757:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545481a0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27836416->27840511] (req 27836416->27840511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.022845:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01377480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495954048:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40871us (42079us total) trans 4294974113 rc 0/0 00000100:00100000:3.0:1789760792.022856:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6825 00000100:00100000:3.0:1789760792.022866:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776266245178 time 776302673324 state 0x2 ref 0 00000100:00100000:2.0:1789760792.030775:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760792.030921:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495954176 00000100:00100000:3.0:1789760792.030932:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495954176 00000100:00100000:3.0:1789760792.030986:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776266245178 time 776302673324 state 0x0 ref 1 00000100:00100000:3.0:1789760792.031002:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776302933740 time 776302673324 state 0x1 ref 1 00000100:00100000:3.0:1789760792.031036:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776302933740 time 776351753478 state 0x0 ref 1 00000100:00100000:3.0:1789760792.031049:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fdf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495954176:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760792.031212:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27840512, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:3.0:1789760792.031221:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481a7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.031236:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760792.031240:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.031246:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27840512->27844607] (req 27840512->27844607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.031259:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27840512->27844607] (req 27840512->27844607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.069963:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974114 is committed 00002000:00010000:3.0:1789760792.070069:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27840512 -> 27844608 00010000:00010000:3.0:1789760792.070086:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27840512->27844607] (req 27840512->27844607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.070098:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481a7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27840512->27844607] (req 27840512->27844607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.070108:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27840512->27844607] (req 27840512->27844607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.070121:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760792.070125:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27840512->27844607] (req 27840512->27844607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.070135:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481a7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27840512->27844607] (req 27840512->27844607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.070159:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481a7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27840512->27844607] (req 27840512->27844607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.070167:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481a7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27840512->27844607] (req 27840512->27844607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.070392:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fdf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495954176:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39340us (39621us total) trans 4294974114 rc 0/0 00000100:00100000:3.0:1789760792.070402:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6826 00000100:00100000:3.0:1789760792.070411:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776302933740 time 776351753478 state 0x2 ref 0 00000100:00100000:2.0:1789760792.078530:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760792.078626:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495954304 00000100:00100000:3.0:1789760792.078636:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495954304 00000100:00100000:3.0:1789760792.078709:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776302933740 time 776351753478 state 0x0 ref 1 00000100:00100000:3.0:1789760792.078725:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776352013894 time 776351753478 state 0x1 ref 1 00000100:00100000:3.0:1789760792.078759:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776352013894 time 776399474083 state 0x0 ref 1 00000100:00100000:3.0:1789760792.078771:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fdea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495954304:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760792.078818:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27844608, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:3.0:1789760792.078827:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481ae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.078842:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760792.078846:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.078853:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27844608->27848703] (req 27844608->27848703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.078866:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27844608->27848703] (req 27844608->27848703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.140271:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974115 is committed 00002000:00010000:0.0:1789760792.140715:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27844608 -> 27848704 00010000:00010000:0.0:1789760792.140732:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27844608->27848703] (req 27844608->27848703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.140744:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481ae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27844608->27848703] (req 27844608->27848703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.140753:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27844608->27848703] (req 27844608->27848703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.140765:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760792.140769:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27844608->27848703] (req 27844608->27848703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.140778:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481ae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27844608->27848703] (req 27844608->27848703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.140802:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481ae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27844608->27848703] (req 27844608->27848703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.140810:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481ae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27844608->27848703] (req 27844608->27848703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.141054:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fdea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495954304:12345-192.168.203.9@tcp:4:dd.0 Request processed in 62280us (62529us total) trans 4294974115 rc 0/0 00000100:00100000:0.0:1789760792.141065:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6827 00000100:00100000:0.0:1789760792.141074:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776352013894 time 776399474083 state 0x2 ref 0 00000100:00100000:1.0:1789760792.147395:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760792.147482:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495954432 00000100:00100000:3.0:1789760792.147492:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495954432 00000100:00100000:3.0:1789760792.147548:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776352013894 time 776399474083 state 0x0 ref 1 00000100:00100000:3.0:1789760792.147564:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776399734499 time 776399474083 state 0x1 ref 1 00000100:00100000:3.0:1789760792.147596:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776399734499 time 776468312658 state 0x0 ref 1 00000100:00100000:3.0:1789760792.147610:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495954432:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760792.147685:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27848704, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:3.0:1789760792.147695:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481b5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.147709:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760792.147714:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.147721:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27848704->27852799] (req 27848704->27852799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.147737:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27848704->27852799] (req 27848704->27852799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.188599:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974116 is committed 00002000:00010000:3.0:1789760792.188740:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27848704 -> 27852800 00010000:00010000:3.0:1789760792.188756:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27848704->27852799] (req 27848704->27852799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.188769:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481b5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27848704->27852799] (req 27848704->27852799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.188779:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27848704->27852799] (req 27848704->27852799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.188788:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760792.188792:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27848704->27852799] (req 27848704->27852799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.188802:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27848704->27852799] (req 27848704->27852799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.188821:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481b5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27848704->27852799] (req 27848704->27852799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.188830:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481b5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27848704->27852799] (req 27848704->27852799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.188921:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495954432:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41309us (41539us total) trans 4294974116 rc 0/0 00000100:00100000:3.0:1789760792.188935:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6828 00000100:00100000:3.0:1789760792.188945:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776399734499 time 776468312658 state 0x2 ref 0 00000100:00100000:0.0:1789760792.192686:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760792.192760:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495954560 00000100:00100000:3.0:1789760792.192769:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495954560 00000100:00100000:3.0:1789760792.192822:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776399734499 time 776468312658 state 0x0 ref 1 00000100:00100000:3.0:1789760792.192837:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776468573074 time 776468312658 state 0x1 ref 1 00000100:00100000:3.0:1789760792.192916:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776468573074 time 776513631873 state 0x0 ref 1 00000100:00100000:3.0:1789760792.192929:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01376680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495954560:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760792.192975:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27852800, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:3.0:1789760792.192984:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545481bc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.193000:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760792.193004:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.193011:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545481bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27852800->27856895] (req 27852800->27856895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.193025:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545481bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27852800->27856895] (req 27852800->27856895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.219562:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974117 is committed 00002000:00010000:3.0:1789760792.219872:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27852800 -> 27856896 00010000:00010000:3.0:1789760792.219888:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545481bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27852800->27856895] (req 27852800->27856895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.219901:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545481bc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27852800->27856895] (req 27852800->27856895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.219910:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545481bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27852800->27856895] (req 27852800->27856895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.219917:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760792.219923:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545481bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27852800->27856895] (req 27852800->27856895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.219933:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545481bc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27852800->27856895] (req 27852800->27856895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.219952:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545481bc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27852800->27856895] (req 27852800->27856895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.219960:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545481bc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27852800->27856895] (req 27852800->27856895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.220089:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01376680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495954560:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27156us (27424us total) trans 4294974117 rc 0/0 00000100:00100000:3.0:1789760792.220101:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6829 00000100:00100000:3.0:1789760792.220112:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776468573074 time 776513631873 state 0x2 ref 0 00000100:00100000:1.0:1789760792.227643:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760792.227752:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495954688 00000100:00100000:0.0:1789760792.227761:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495954688 00000100:00100000:0.0:1789760792.227816:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776468573074 time 776513631873 state 0x0 ref 1 00000100:00100000:0.0:1789760792.227831:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776513892289 time 776513631873 state 0x1 ref 1 00000100:00100000:0.0:1789760792.227859:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776513892289 time 776548575337 state 0x0 ref 1 00000100:00100000:0.0:1789760792.227871:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02768e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495954688:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760792.227915:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27856896, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:0.0:1789760792.227925:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545481c3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.227948:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:0.0:1789760792.227953:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.227962:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545481c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27856896->27860991] (req 27856896->27860991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.227977:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545481c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27856896->27860991] (req 27856896->27860991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.251610:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974118 is committed 00002000:00010000:0.0:1789760792.251782:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27856896 -> 27860992 00010000:00010000:0.0:1789760792.251798:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545481c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27856896->27860991] (req 27856896->27860991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.251811:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545481c3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27856896->27860991] (req 27856896->27860991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.251820:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545481c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27856896->27860991] (req 27856896->27860991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.251828:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760792.251833:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545481c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27856896->27860991] (req 27856896->27860991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.251842:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545481c3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27856896->27860991] (req 27856896->27860991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.251860:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545481c3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27856896->27860991] (req 27856896->27860991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.251868:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545481c3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27856896->27860991] (req 27856896->27860991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.251952:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02768e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495954688:12345-192.168.203.9@tcp:4:dd.0 Request processed in 24078us (24315us total) trans 4294974118 rc 0/0 00000100:00100000:0.0:1789760792.251961:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6830 00000100:00100000:0.0:1789760792.251970:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776513892289 time 776548575337 state 0x2 ref 0 00000100:00100000:1.0:1789760792.255564:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760792.255700:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495954816 00000100:00100000:3.0:1789760792.255711:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495954816 00000100:00100000:3.0:1789760792.255771:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776513892289 time 776548575337 state 0x0 ref 1 00000100:00100000:3.0:1789760792.255786:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776548835753 time 776548575337 state 0x1 ref 1 00000100:00100000:3.0:1789760792.255870:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776548835753 time 776576585792 state 0x0 ref 1 00000100:00100000:3.0:1789760792.255882:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495954816:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760792.255928:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27860992, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:3.0:1789760792.255937:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545481ca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.255953:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760792.255958:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.255964:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545481ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27860992->27865087] (req 27860992->27865087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.255978:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545481ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27860992->27865087] (req 27860992->27865087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.281555:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974119 is committed 00002000:00010000:2.0:1789760792.281617:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27860992 -> 27865088 00010000:00010000:2.0:1789760792.281632:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545481ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27860992->27865087] (req 27860992->27865087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.281672:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545481ca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27860992->27865087] (req 27860992->27865087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.281681:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545481ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27860992->27865087] (req 27860992->27865087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.281688:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760792.281694:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545481ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27860992->27865087] (req 27860992->27865087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.281702:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545481ca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27860992->27865087] (req 27860992->27865087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.281721:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545481ca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27860992->27865087] (req 27860992->27865087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.281729:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545481ca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27860992->27865087] (req 27860992->27865087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.281821:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495954816:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25936us (26262us total) trans 4294974119 rc 0/0 00000100:00100000:2.0:1789760792.281832:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6831 00000100:00100000:2.0:1789760792.281843:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776548835753 time 776576585792 state 0x2 ref 0 00000100:00100000:0.0:1789760792.286331:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760792.286367:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495954944 00000100:00100000:3.0:1789760792.286377:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495954944 00000100:00100000:3.0:1789760792.286435:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776548835753 time 776576585792 state 0x0 ref 1 00000100:00100000:3.0:1789760792.286449:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776576846208 time 776576585792 state 0x1 ref 1 00000100:00100000:3.0:1789760792.286480:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776576846208 time 776607197076 state 0x0 ref 1 00000100:00100000:3.0:1789760792.286495:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01376d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495954944:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760792.286539:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27865088, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:3.0:1789760792.286547:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481d1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.286564:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760792.286569:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.286575:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27865088->27869183] (req 27865088->27869183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.286589:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27865088->27869183] (req 27865088->27869183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.317183:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974120 is committed 00002000:00010000:1.0:1789760792.317613:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27865088 -> 27869184 00010000:00010000:1.0:1789760792.317629:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27865088->27869183] (req 27865088->27869183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.317710:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481d1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27865088->27869183] (req 27865088->27869183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.317720:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27865088->27869183] (req 27865088->27869183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.317728:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760792.317733:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27865088->27869183] (req 27865088->27869183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.317742:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481d1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27865088->27869183] (req 27865088->27869183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.317761:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481d1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27865088->27869183] (req 27865088->27869183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.317769:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545481d1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27865088->27869183] (req 27865088->27869183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.317861:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01376d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495954944:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31364us (31582us total) trans 4294974120 rc 0/0 00000100:00100000:1.0:1789760792.317872:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6832 00000100:00100000:1.0:1789760792.317882:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776576846208 time 776607197076 state 0x2 ref 0 00000100:00100000:1.0:1789760792.321450:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760792.321536:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495955072 00000100:00100000:0.0:1789760792.321547:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495955072 00000100:00100000:0.0:1789760792.321601:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776576846208 time 776607197076 state 0x0 ref 1 00000100:00100000:0.0:1789760792.321618:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776607457492 time 776607197076 state 0x1 ref 1 00000100:00100000:0.0:1789760792.321781:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776607457492 time 776642497086 state 0x0 ref 1 00000100:00100000:0.0:1789760792.321794:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495955072:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760792.321840:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27869184, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:0.0:1789760792.321849:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545481d8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.321866:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:0.0:1789760792.321871:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.321878:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545481d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27869184->27873279] (req 27869184->27873279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.321892:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545481d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27869184->27873279] (req 27869184->27873279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.378919:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974121 is committed 00002000:00010000:1.0:1789760792.379108:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27869184 -> 27873280 00010000:00010000:1.0:1789760792.379124:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545481d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27869184->27873279] (req 27869184->27873279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.379137:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545481d8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27869184->27873279] (req 27869184->27873279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.379146:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545481d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27869184->27873279] (req 27869184->27873279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.379154:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760792.379159:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545481d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27869184->27873279] (req 27869184->27873279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.379168:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545481d8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27869184->27873279] (req 27869184->27873279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.379188:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545481d8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27869184->27873279] (req 27869184->27873279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.379196:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x49367483545481d8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27869184->27873279] (req 27869184->27873279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.379322:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495955072:12345-192.168.203.9@tcp:4:dd.0 Request processed in 57488us (57840us total) trans 4294974121 rc 0/0 00000100:00100000:1.0:1789760792.379335:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6833 00000100:00100000:1.0:1789760792.379344:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776607457492 time 776642497086 state 0x2 ref 0 00000100:00100000:3.0:1789760792.388586:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760792.388931:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495955200 00000100:00100000:2.0:1789760792.388941:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495955200 00000100:00100000:2.0:1789760792.388997:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776607457492 time 776642497086 state 0x0 ref 1 00000100:00100000:2.0:1789760792.389014:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776642757502 time 776642497086 state 0x1 ref 1 00000100:00100000:2.0:1789760792.389104:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776642757502 time 776709820209 state 0x0 ref 1 00000100:00100000:2.0:1789760792.389117:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495955200:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760792.389166:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27873280, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:2.0:1789760792.389176:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545481df lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.389191:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760792.389196:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.389203:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545481df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27873280->27877375] (req 27873280->27877375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.389229:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545481df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27873280->27877375] (req 27873280->27877375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.414924:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760792.446342:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974122 is committed 00002000:00010000:0.0:1789760792.446449:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27873280 -> 27877376 00010000:00010000:0.0:1789760792.446465:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545481df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27873280->27877375] (req 27873280->27877375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.446478:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545481df lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27873280->27877375] (req 27873280->27877375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.446487:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545481df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27873280->27877375] (req 27873280->27877375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.446494:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760792.446499:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545481df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27873280->27877375] (req 27873280->27877375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.446508:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545481df lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27873280->27877375] (req 27873280->27877375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.446528:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545481df lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27873280->27877375] (req 27873280->27877375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.446536:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x49367483545481df lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27873280->27877375] (req 27873280->27877375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.447249:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495955200:12345-192.168.203.9@tcp:4:dd.0 Request processed in 58127us (58666us total) trans 4294974122 rc 0/0 00000100:00100000:0.0:1789760792.447259:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6834 00000100:00100000:0.0:1789760792.447269:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776642757502 time 776709820209 state 0x2 ref 0 00000100:00100000:2.0:1789760792.451584:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760792.451755:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495955328 00000100:00100000:1.0:1789760792.451764:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495955328 00000100:00100000:1.0:1789760792.451814:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776642757502 time 776709820209 state 0x0 ref 1 00000100:00100000:1.0:1789760792.451829:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776710080625 time 776709820209 state 0x1 ref 1 00000100:00100000:1.0:1789760792.451857:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776710080625 time 776772573620 state 0x0 ref 1 00000100:00100000:1.0:1789760792.451869:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fdf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495955328:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760792.451917:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27877376, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:1.0:1789760792.451926:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545481e6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.451943:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760792.451948:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.451954:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545481e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27877376->27881471] (req 27877376->27881471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.451967:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545481e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27877376->27881471] (req 27877376->27881471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.479500:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974123 is committed 00002000:00010000:3.0:1789760792.479585:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27877376 -> 27881472 00010000:00010000:3.0:1789760792.479601:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545481e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27877376->27881471] (req 27877376->27881471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.479615:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545481e6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27877376->27881471] (req 27877376->27881471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.479774:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545481e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27877376->27881471] (req 27877376->27881471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.479785:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760792.479791:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545481e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27877376->27881471] (req 27877376->27881471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.479801:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545481e6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27877376->27881471] (req 27877376->27881471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.479823:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545481e6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27877376->27881471] (req 27877376->27881471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.479831:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x49367483545481e6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27877376->27881471] (req 27877376->27881471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.480042:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fdf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495955328:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28170us (28461us total) trans 4294974123 rc 0/0 00000100:00100000:3.0:1789760792.480053:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6835 00000100:00100000:3.0:1789760792.480062:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776710080625 time 776772573620 state 0x2 ref 0 00000100:00100000:1.0:1789760792.489591:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760792.489719:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495955456 00000100:00100000:0.0:1789760792.489734:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495955456 00000100:00100000:0.0:1789760792.489797:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776710080625 time 776772573620 state 0x0 ref 1 00000100:00100000:0.0:1789760792.489812:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776772834036 time 776772573620 state 0x1 ref 1 00000100:00100000:0.0:1789760792.489836:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776772834036 time 776810553157 state 0x0 ref 1 00000100:00100000:0.0:1789760792.489849:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495955456:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760792.489902:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27881472, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:0.0:1789760792.489912:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545481ed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.489928:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:0.0:1789760792.489933:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.489939:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545481ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27881472->27885567] (req 27881472->27885567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.489958:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545481ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27881472->27885567] (req 27881472->27885567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760792.525271:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27881472 -> 27885568 00010000:00010000:2.0:1789760792.525389:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545481ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27881472->27885567] (req 27881472->27885567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.525402:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545481ed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27881472->27885567] (req 27881472->27885567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.525410:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545481ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27881472->27885567] (req 27881472->27885567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.525417:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760792.525422:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545481ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27881472->27885567] (req 27881472->27885567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.525431:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545481ed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27881472->27885567] (req 27881472->27885567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.525452:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545481ed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27881472->27885567] (req 27881472->27885567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.525459:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545481ed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27881472->27885567] (req 27881472->27885567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.525551:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495955456:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35700us (35965us total) trans 4294974124 rc 0/0 00000100:00100000:2.0:1789760792.525562:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6836 00000100:00100000:2.0:1789760792.525573:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776772834036 time 776810553157 state 0x2 ref 0 00000001:00080000:3.0:1789760792.525764:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974124 is committed 00000100:00100000:1.0:1789760792.535222:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760792.535472:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495955584 00000100:00100000:0.0:1789760792.535482:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495955584 00000100:00100000:1.0:1789760792.535612:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776772834036 time 776810553157 state 0x0 ref 1 00000100:00100000:1.0:1789760792.535627:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776810813573 time 776810553157 state 0x1 ref 1 00000100:00100000:1.0:1789760792.535896:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776810813573 time 776856601512 state 0x0 ref 1 00000100:00100000:1.0:1789760792.535911:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495955584:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760792.535969:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27885568, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:1.0:1789760792.535978:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545481f4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.535996:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760792.536001:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.536009:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545481f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27885568->27889663] (req 27885568->27889663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.536023:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545481f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27885568->27889663] (req 27885568->27889663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.582965:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974125 is committed 00002000:00010000:1.0:1789760792.583050:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27885568 -> 27889664 00010000:00010000:1.0:1789760792.583066:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545481f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27885568->27889663] (req 27885568->27889663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.583078:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545481f4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27885568->27889663] (req 27885568->27889663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.583087:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545481f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27885568->27889663] (req 27885568->27889663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.583096:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760792.583101:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545481f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27885568->27889663] (req 27885568->27889663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.583110:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545481f4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27885568->27889663] (req 27885568->27889663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.583131:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545481f4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27885568->27889663] (req 27885568->27889663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.585451:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545481f4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27885568->27889663] (req 27885568->27889663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.585935:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495955584:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50022us (50715us total) trans 4294974125 rc 0/0 00000100:00100000:3.0:1789760792.585947:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6837 00000100:00100000:3.0:1789760792.585957:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776810813573 time 776856601512 state 0x2 ref 0 00000100:00100000:0.0:1789760792.606124:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760792.606406:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495955712 00000100:00100000:3.0:1789760792.606416:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495955712 00000100:00100000:2.0:1789760792.606957:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776810813573 time 776856601512 state 0x0 ref 1 00000100:00100000:2.0:1789760792.606973:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776856861928 time 776856601512 state 0x1 ref 1 00000100:00100000:2.0:1789760792.607009:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776856861928 time 776927725410 state 0x0 ref 1 00000100:00100000:2.0:1789760792.607022:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01374000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495955712:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760792.607067:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27889664, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:2.0:1789760792.607076:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545481fb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.607093:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760792.607102:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.607109:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545481fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27889664->27893759] (req 27889664->27893759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.607122:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545481fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27889664->27893759] (req 27889664->27893759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.644494:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974126 is committed 00002000:00010000:3.0:1789760792.644615:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27889664 -> 27893760 00010000:00010000:3.0:1789760792.644629:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545481fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27889664->27893759] (req 27889664->27893759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.644655:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545481fb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27889664->27893759] (req 27889664->27893759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.644673:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545481fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27889664->27893759] (req 27889664->27893759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.644681:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760792.644686:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545481fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27889664->27893759] (req 27889664->27893759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.644694:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545481fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27889664->27893759] (req 27889664->27893759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.644714:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545481fb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27889664->27893759] (req 27889664->27893759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.644722:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545481fb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27889664->27893759] (req 27889664->27893759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.644833:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01374000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495955712:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37810us (38717us total) trans 4294974126 rc 0/0 00000100:00100000:3.0:1789760792.644843:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6838 00000100:00100000:3.0:1789760792.644857:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776856861928 time 776927725410 state 0x2 ref 0 00000100:00100000:1.0:1789760792.650760:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760792.652137:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495955840 00000100:00100000:2.0:1789760792.652147:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495955840 00000100:00100000:2.0:1789760792.652201:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776856861928 time 776927725410 state 0x0 ref 1 00000100:00100000:2.0:1789760792.652217:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776927985826 time 776927725410 state 0x1 ref 1 00000100:00100000:2.0:1789760792.652298:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776927985826 time 776973013844 state 0x0 ref 1 00000100:00100000:2.0:1789760792.652311:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495955840:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760792.652357:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27893760, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:2.0:1789760792.652366:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354548202 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.652381:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760792.652385:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.652392:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354548202 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27893760->27897855] (req 27893760->27897855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.652405:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354548202 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27893760->27897855] (req 27893760->27897855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.680856:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974127 is committed 00002000:00010000:3.0:1789760792.680895:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27893760 -> 27897856 00002000:00010000:3.0:1789760792.680900:0:13059:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760791 -> 1789760792 00002000:00010000:3.0:1789760792.680903:0:13059:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760791 -> 1789760792 00010000:00010000:3.0:1789760792.680917:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354548202 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27893760->27897855] (req 27893760->27897855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.680931:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354548202 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27893760->27897855] (req 27893760->27897855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.680941:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354548202 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27893760->27897855] (req 27893760->27897855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.680949:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760792.680954:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354548202 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27893760->27897855] (req 27893760->27897855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.680963:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354548202 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27893760->27897855] (req 27893760->27897855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.681106:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354548202 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27893760->27897855] (req 27893760->27897855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.681113:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354548202 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27893760->27897855] (req 27893760->27897855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.681211:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495955840:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28894us (30451us total) trans 4294974127 rc 0/0 00000100:00100000:3.0:1789760792.681221:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6839 00000100:00100000:3.0:1789760792.681232:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776927985826 time 776973013844 state 0x2 ref 0 00000100:00100000:1.0:1789760792.686868:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760792.686965:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495955968 00000100:00100000:3.0:1789760792.686974:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495955968 00000100:00100000:3.0:1789760792.687118:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776927985826 time 776973013844 state 0x0 ref 1 00000100:00100000:3.0:1789760792.687134:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776973274260 time 776973013844 state 0x1 ref 1 00000100:00100000:3.0:1789760792.687157:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776973274260 time 777007873841 state 0x0 ref 1 00000100:00100000:3.0:1789760792.687168:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495955968:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760792.687226:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27897856, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:3.0:1789760792.687236:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548209 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.687255:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760792.687260:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.687266:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548209 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27897856->27901951] (req 27897856->27901951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.687306:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548209 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27897856->27901951] (req 27897856->27901951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.719421:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974128 is committed 00002000:00010000:3.0:1789760792.719519:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27897856 -> 27901952 00010000:00010000:3.0:1789760792.719535:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548209 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27897856->27901951] (req 27897856->27901951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.719552:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548209 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27897856->27901951] (req 27897856->27901951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.719562:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548209 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27897856->27901951] (req 27897856->27901951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.719569:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760792.719575:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548209 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27897856->27901951] (req 27897856->27901951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.719584:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548209 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27897856->27901951] (req 27897856->27901951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.719605:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548209 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27897856->27901951] (req 27897856->27901951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.719614:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548209 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27897856->27901951] (req 27897856->27901951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.719717:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495955968:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32545us (32853us total) trans 4294974128 rc 0/0 00000100:00100000:3.0:1789760792.719727:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6840 00000100:00100000:3.0:1789760792.719736:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776973274260 time 777007873841 state 0x2 ref 0 00000100:00100000:0.0:1789760792.722868:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760792.722971:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495956096 00000100:00100000:3.0:1789760792.723060:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495956096 00000100:00100000:3.0:1789760792.723110:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 776973274260 time 777007873841 state 0x0 ref 1 00000100:00100000:3.0:1789760792.723124:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777008134257 time 777007873841 state 0x1 ref 1 00000100:00100000:3.0:1789760792.723154:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777008134257 time 777043871699 state 0x0 ref 1 00000100:00100000:3.0:1789760792.723166:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495956096:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760792.723211:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27901952, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:3.0:1789760792.723220:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548210 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.723235:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760792.723240:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.723246:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548210 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27901952->27906047] (req 27901952->27906047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.723260:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548210 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27901952->27906047] (req 27901952->27906047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760792.764886:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27901952 -> 27906048 00010000:00010000:3.0:1789760792.764903:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548210 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27901952->27906047] (req 27901952->27906047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.764917:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548210 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27901952->27906047] (req 27901952->27906047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.764932:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548210 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27901952->27906047] (req 27901952->27906047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.764939:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760792.764944:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548210 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27901952->27906047] (req 27901952->27906047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.764953:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548210 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27901952->27906047] (req 27901952->27906047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.764971:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548210 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27901952->27906047] (req 27901952->27906047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.764988:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548210 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27901952->27906047] (req 27901952->27906047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.765070:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495956096:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41901us (42205us total) trans 4294974129 rc 0/0 00000100:00100000:3.0:1789760792.765080:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6841 00000100:00100000:3.0:1789760792.765090:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777008134257 time 777043871699 state 0x2 ref 0 00000001:00080000:2.0:1789760792.766917:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974129 is committed 00000100:00100000:3.0:1789760792.768051:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760792.768126:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495956224 00000100:00100000:2.0:1789760792.768135:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495956224 00000100:00100000:2.0:1789760792.768184:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777008134257 time 777043871699 state 0x0 ref 1 00000100:00100000:2.0:1789760792.768199:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777044132115 time 777043871699 state 0x1 ref 1 00000100:00100000:2.0:1789760792.768228:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777044132115 time 777088944972 state 0x0 ref 1 00000100:00100000:2.0:1789760792.768240:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495956224:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760792.768305:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27906048, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:2.0:1789760792.768314:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548217 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.768335:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=777s v=(0 0) 00010000:00010000:2.0:1789760792.768339:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.768345:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548217 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27906048->27910143] (req 27906048->27910143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.768358:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548217 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27906048->27910143] (req 27906048->27910143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760792.801037:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27906048 -> 27910144 00010000:00010000:2.0:1789760792.801058:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548217 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27906048->27910143] (req 27906048->27910143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.801070:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548217 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27906048->27910143] (req 27906048->27910143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.801079:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548217 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27906048->27910143] (req 27906048->27910143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.801087:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760792.801092:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548217 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27906048->27910143] (req 27906048->27910143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.801102:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548217 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27906048->27910143] (req 27906048->27910143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.801123:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548217 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27906048->27910143] (req 27906048->27910143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.801133:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548217 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27906048->27910143] (req 27906048->27910143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.801223:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495956224:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32980us (33173us total) trans 4294974130 rc 0/0 00000100:00100000:2.0:1789760792.801234:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6842 00000100:00100000:2.0:1789760792.801244:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777044132115 time 777088944972 state 0x2 ref 0 00000001:00080000:0.0:1789760792.804961:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974130 is committed 00000100:00100000:1.0:1789760792.807571:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760792.807851:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495956352 00000100:00100000:3.0:1789760792.807861:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495956352 00000100:00100000:3.0:1789760792.807916:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777044132115 time 777088944972 state 0x0 ref 1 00000100:00100000:3.0:1789760792.807935:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777089205388 time 777088944972 state 0x1 ref 1 00000100:00100000:3.0:1789760792.808009:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777089205388 time 777128724128 state 0x0 ref 1 00000100:00100000:3.0:1789760792.808023:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495956352:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760792.808073:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27910144, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:3.0:1789760792.808083:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454821e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.808099:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760792.808104:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.808111:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454821e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27910144->27914239] (req 27910144->27914239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.808126:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454821e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27910144->27914239] (req 27910144->27914239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.840127:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974131 is committed 00002000:00010000:2.0:1789760792.840218:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27910144 -> 27914240 00010000:00010000:2.0:1789760792.840235:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454821e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27910144->27914239] (req 27910144->27914239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.840249:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454821e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27910144->27914239] (req 27910144->27914239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.840258:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454821e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27910144->27914239] (req 27910144->27914239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.840265:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760792.840270:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454821e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27910144->27914239] (req 27910144->27914239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.840296:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454821e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27910144->27914239] (req 27910144->27914239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.840321:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454821e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27910144->27914239] (req 27910144->27914239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.840329:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454821e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27910144->27914239] (req 27910144->27914239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.840415:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495956352:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32391us (32849us total) trans 4294974131 rc 0/0 00000100:00100000:2.0:1789760792.840425:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6843 00000100:00100000:2.0:1789760792.840434:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777089205388 time 777128724128 state 0x2 ref 0 00000100:00100000:1.0:1789760792.845680:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760792.845874:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495956480 00000100:00100000:3.0:1789760792.845884:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495956480 00000100:00100000:3.0:1789760792.845932:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777089205388 time 777128724128 state 0x0 ref 1 00000100:00100000:3.0:1789760792.845947:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777128984544 time 777128724128 state 0x1 ref 1 00000100:00100000:3.0:1789760792.845987:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777128984544 time 777166703414 state 0x0 ref 1 00000100:00100000:3.0:1789760792.845998:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495956480:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760792.846041:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27914240, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:3.0:1789760792.846050:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548225 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.846066:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760792.846070:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.846076:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548225 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27914240->27918335] (req 27914240->27918335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.846090:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548225 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27914240->27918335] (req 27914240->27918335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.887894:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974132 is committed 00002000:00010000:2.0:1789760792.888042:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27914240 -> 27918336 00010000:00010000:2.0:1789760792.888058:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548225 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27914240->27918335] (req 27914240->27918335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.888071:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548225 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27914240->27918335] (req 27914240->27918335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.888080:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548225 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27914240->27918335] (req 27914240->27918335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.888092:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760792.888098:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548225 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27914240->27918335] (req 27914240->27918335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.888113:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548225 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27914240->27918335] (req 27914240->27918335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.888133:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548225 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27914240->27918335] (req 27914240->27918335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.888141:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548225 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27914240->27918335] (req 27914240->27918335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.888232:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495956480:12345-192.168.203.9@tcp:4:dd.0 Request processed in 42230us (42556us total) trans 4294974132 rc 0/0 00000100:00100000:2.0:1789760792.888242:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6844 00000100:00100000:2.0:1789760792.888251:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777128984544 time 777166703414 state 0x2 ref 0 00000100:00100000:0.0:1789760792.894149:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760792.894860:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495956608 00000100:00100000:2.0:1789760792.894871:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495956608 00000100:00100000:2.0:1789760792.894927:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777128984544 time 777166703414 state 0x0 ref 1 00000100:00100000:2.0:1789760792.894944:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777166963830 time 777166703414 state 0x1 ref 1 00000100:00100000:0.0:1789760792.895312:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777166963830 time 777216025368 state 0x0 ref 1 00000100:00100000:0.0:1789760792.895329:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01374700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495956608:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760792.895378:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27918336, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:0.0:1789760792.895388:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454822c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.895405:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=777s v=(0 0) 00010000:00010000:0.0:1789760792.895409:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.895427:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454822c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27918336->27922431] (req 27918336->27922431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.895442:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454822c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27918336->27922431] (req 27918336->27922431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760792.936830:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27918336 -> 27922432 00010000:00010000:2.0:1789760792.936851:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454822c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27918336->27922431] (req 27918336->27922431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.936864:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454822c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27918336->27922431] (req 27918336->27922431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.936879:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454822c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27918336->27922431] (req 27918336->27922431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.936887:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760792.936891:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454822c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27918336->27922431] (req 27918336->27922431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.936900:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454822c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27918336->27922431] (req 27918336->27922431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.936921:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454822c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27918336->27922431] (req 27918336->27922431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.936929:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec5c5f6e/0x493674835454822c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27918336->27922431] (req 27918336->27922431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.937031:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01374700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495956608:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41702us (42885us total) trans 4294974133 rc 0/0 00000100:00100000:2.0:1789760792.937042:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6845 00000100:00100000:2.0:1789760792.937052:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777166963830 time 777216025368 state 0x2 ref 0 00000001:00080000:0.0:1789760792.937535:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974133 is committed 00000100:00100000:1.0:1789760792.940462:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760792.940545:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495956736 00000100:00100000:2.0:1789760792.940556:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495956736 00000100:00100000:2.0:1789760792.940609:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777166963830 time 777216025368 state 0x0 ref 1 00000100:00100000:2.0:1789760792.940624:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777216285784 time 777216025368 state 0x1 ref 1 00000100:00100000:2.0:1789760792.940673:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777216285784 time 777261389654 state 0x0 ref 1 00000100:00100000:2.0:1789760792.940686:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495956736:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760792.940732:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27922432, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:2.0:1789760792.940741:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548233 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.940755:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=777s v=(0 0) 00010000:00010000:2.0:1789760792.940759:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.940765:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548233 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27922432->27926527] (req 27922432->27926527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.940778:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548233 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27922432->27926527] (req 27922432->27926527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.977665:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974134 is committed 00002000:00010000:3.0:1789760792.977802:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27922432 -> 27926528 00010000:00010000:3.0:1789760792.977819:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548233 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27922432->27926527] (req 27922432->27926527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.977833:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548233 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27922432->27926527] (req 27922432->27926527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.977841:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548233 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27922432->27926527] (req 27922432->27926527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.977848:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760792.977852:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548233 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27922432->27926527] (req 27922432->27926527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.977861:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548233 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27922432->27926527] (req 27922432->27926527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.977880:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548233 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27922432->27926527] (req 27922432->27926527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.977887:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x4936748354548233 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27922432->27926527] (req 27922432->27926527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.978119:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495956736:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37429us (37660us total) trans 4294974134 rc 0/0 00000100:00100000:3.0:1789760792.978130:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6846 00000100:00100000:3.0:1789760792.978140:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777216285784 time 777261389654 state 0x2 ref 0 00000100:00100000:1.0:1789760792.983856:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760792.983945:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495956864 00000100:00100000:2.0:1789760792.983956:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495956864 00000100:00100000:2.0:1789760792.984025:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777216285784 time 777261389654 state 0x0 ref 1 00000100:00100000:2.0:1789760792.984041:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777261650070 time 777261389654 state 0x1 ref 1 00000100:00100000:2.0:1789760792.984073:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777261650070 time 777304790042 state 0x0 ref 1 00000100:00100000:2.0:1789760792.984086:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495956864:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760792.984133:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27926528, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:2.0:1789760792.984143:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454823a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.984158:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=777s v=(0 0) 00010000:00010000:2.0:1789760792.984163:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.984169:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454823a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27926528->27930623] (req 27926528->27930623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.984183:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454823a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27926528->27930623] (req 27926528->27930623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.015007:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974135 is committed 00002000:00010000:1.0:1789760793.015165:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27926528 -> 27930624 00010000:00010000:1.0:1789760793.015180:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454823a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27926528->27930623] (req 27926528->27930623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.015193:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454823a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27926528->27930623] (req 27926528->27930623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.015203:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454823a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27926528->27930623] (req 27926528->27930623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.015221:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760793.015227:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454823a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27926528->27930623] (req 27926528->27930623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.015237:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454823a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27926528->27930623] (req 27926528->27930623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.015256:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454823a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27926528->27930623] (req 27926528->27930623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.015263:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x493674835454823a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27926528->27930623] (req 27926528->27930623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.015383:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495956864:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31295us (31533us total) trans 4294974135 rc 0/0 00000100:00100000:1.0:1789760793.015393:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6847 00000100:00100000:1.0:1789760793.015404:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777261650070 time 777304790042 state 0x2 ref 0 00000100:00100000:0.0:1789760793.020955:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760793.028899:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495956992 00000100:00100000:2.0:1789760793.028915:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495956992 00000100:00100000:2.0:1789760793.029004:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777261650070 time 777304790042 state 0x0 ref 1 00000100:00100000:2.0:1789760793.029024:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777305050458 time 777304790042 state 0x1 ref 1 00000100:00080000:0.0:1789760793.029117:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760793.029682:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777305050458 time 777350395406 state 0x0 ref 1 00000100:00100000:2.0:1789760793.029701:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01375c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495956992:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760793.029760:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27930624, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:2.0:1789760793.029769:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354548241 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.029786:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=777s v=(0 0) 00010000:00010000:2.0:1789760793.029790:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.029798:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354548241 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27930624->27934719] (req 27930624->27934719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.029811:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354548241 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27930624->27934719] (req 27930624->27934719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.079266:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974136 is committed 00002000:00010000:1.0:1789760793.079464:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27930624 -> 27934720 00010000:00010000:1.0:1789760793.079480:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354548241 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27930624->27934719] (req 27930624->27934719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.079500:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354548241 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27930624->27934719] (req 27930624->27934719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.079510:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354548241 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27930624->27934719] (req 27930624->27934719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.079517:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760793.079522:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354548241 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27930624->27934719] (req 27930624->27934719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.079532:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354548241 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27930624->27934719] (req 27930624->27934719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.079552:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354548241 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27930624->27934719] (req 27930624->27934719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.079561:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x4936748354548241 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27930624->27934719] (req 27930624->27934719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.079683:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01375c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495956992:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49984us (58732us total) trans 4294974136 rc 0/0 00000100:00100000:1.0:1789760793.079694:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6848 00000100:00100000:1.0:1789760793.079704:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777305050458 time 777350395406 state 0x2 ref 0 00000100:00100000:1.0:1789760793.083698:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760793.083795:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495957120 00000100:00100000:3.0:1789760793.083806:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495957120 00000100:00100000:3.0:1789760793.083871:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777305050458 time 777350395406 state 0x0 ref 1 00000100:00100000:3.0:1789760793.083901:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777350655822 time 777350395406 state 0x1 ref 1 00000100:00100000:3.0:1789760793.083937:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777350655822 time 777404652998 state 0x0 ref 1 00000100:00100000:3.0:1789760793.083949:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495957120:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760793.083999:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27934720, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:3.0:1789760793.084008:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354548248 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.084024:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760793.084030:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.084037:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354548248 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27934720->27938815] (req 27934720->27938815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.084052:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354548248 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27934720->27938815] (req 27934720->27938815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760793.126852:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27934720 -> 27938816 00010000:00010000:0.0:1789760793.126884:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354548248 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27934720->27938815] (req 27934720->27938815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.126898:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354548248 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27934720->27938815] (req 27934720->27938815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.126907:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354548248 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27934720->27938815] (req 27934720->27938815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.126914:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760793.126919:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354548248 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27934720->27938815] (req 27934720->27938815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.126929:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354548248 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27934720->27938815] (req 27934720->27938815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.126950:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354548248 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27934720->27938815] (req 27934720->27938815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.126958:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x4936748354548248 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27934720->27938815] (req 27934720->27938815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.127187:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495957120:12345-192.168.203.9@tcp:4:dd.0 Request processed in 43234us (43496us total) trans 4294974137 rc 0/0 00000100:00100000:0.0:1789760793.127201:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6849 00000100:00100000:0.0:1789760793.127210:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777350655822 time 777404652998 state 0x2 ref 0 00000100:00100000:2.0:1789760793.130762:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760793.130925:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495957248 00000100:00100000:3.0:1789760793.130938:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495957248 00000100:00100000:3.0:1789760793.131010:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777350655822 time 777404652998 state 0x0 ref 1 00000100:00100000:3.0:1789760793.131030:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777404913414 time 777404652998 state 0x1 ref 1 00000100:00100000:3.0:1789760793.131065:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777404913414 time 777451781603 state 0x0 ref 1 00000100:00100000:3.0:1789760793.131077:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fdd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495957248:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760793.131132:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27938816, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:3.0:1789760793.131142:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454824f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.131157:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760793.131162:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.131168:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454824f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27938816->27942911] (req 27938816->27942911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.131184:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454824f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27938816->27942911] (req 27938816->27942911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.132209:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974137 is committed 00000001:00080000:0.0:1789760793.167870:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974138 is committed 00002000:00010000:3.0:1789760793.167984:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27938816 -> 27942912 00010000:00010000:3.0:1789760793.168000:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454824f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27938816->27942911] (req 27938816->27942911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.168013:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454824f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27938816->27942911] (req 27938816->27942911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.168022:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454824f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27938816->27942911] (req 27938816->27942911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.168029:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760793.168034:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454824f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27938816->27942911] (req 27938816->27942911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.168044:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454824f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27938816->27942911] (req 27938816->27942911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.168064:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454824f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27938816->27942911] (req 27938816->27942911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.168073:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x493674835454824f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27938816->27942911] (req 27938816->27942911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.168172:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fdd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495957248:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37089us (37413us total) trans 4294974138 rc 0/0 00000100:00100000:3.0:1789760793.168188:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6850 00000100:00100000:3.0:1789760793.168198:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777404913414 time 777451781603 state 0x2 ref 0 00000100:00100000:3.0:1789760793.173386:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760793.173812:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495957376 00000100:00100000:1.0:1789760793.173822:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495957376 00000100:00100000:1.0:1789760793.173876:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777404913414 time 777451781603 state 0x0 ref 1 00000100:00100000:1.0:1789760793.173891:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777452042019 time 777451781603 state 0x1 ref 1 00000100:00100000:1.0:1789760793.173923:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777452042019 time 777494639806 state 0x0 ref 1 00000100:00100000:1.0:1789760793.173935:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495957376:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760793.173995:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27942912, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:1.0:1789760793.174004:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548256 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.174022:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=777s v=(0 0) 00010000:00010000:1.0:1789760793.174026:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.174033:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548256 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27942912->27947007] (req 27942912->27947007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.174047:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548256 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27942912->27947007] (req 27942912->27947007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.196914:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974139 is committed 00002000:00010000:1.0:1789760793.197017:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27942912 -> 27947008 00010000:00010000:1.0:1789760793.197034:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548256 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27942912->27947007] (req 27942912->27947007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.197046:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548256 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27942912->27947007] (req 27942912->27947007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.197055:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548256 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27942912->27947007] (req 27942912->27947007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.197062:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760793.197067:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548256 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27942912->27947007] (req 27942912->27947007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.197074:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548256 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27942912->27947007] (req 27942912->27947007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.197092:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548256 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27942912->27947007] (req 27942912->27947007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.197100:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548256 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27942912->27947007] (req 27942912->27947007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.197198:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495957376:12345-192.168.203.9@tcp:4:dd.0 Request processed in 23259us (23816us total) trans 4294974139 rc 0/0 00000100:00100000:1.0:1789760793.197208:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6851 00000100:00100000:1.0:1789760793.197217:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777452042019 time 777494639806 state 0x2 ref 0 00000100:00100000:3.0:1789760793.200947:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760793.201053:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495957504 00000100:00100000:0.0:1789760793.201063:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495957504 00000100:00100000:0.0:1789760793.201117:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777452042019 time 777494639806 state 0x0 ref 1 00000100:00100000:0.0:1789760793.201132:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777494900222 time 777494639806 state 0x1 ref 1 00000100:00100000:0.0:1789760793.201159:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777494900222 time 777521875539 state 0x0 ref 1 00000100:00100000:0.0:1789760793.201171:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495957504:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760793.201216:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27947008, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:0.0:1789760793.201225:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454825d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.201245:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=777s v=(0 0) 00010000:00010000:0.0:1789760793.201250:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.201257:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454825d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27947008->27951103] (req 27947008->27951103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.201271:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454825d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27947008->27951103] (req 27947008->27951103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760793.236909:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27947008 -> 27951104 00010000:00010000:1.0:1789760793.236929:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454825d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27947008->27951103] (req 27947008->27951103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.236947:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454825d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27947008->27951103] (req 27947008->27951103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.236957:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454825d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27947008->27951103] (req 27947008->27951103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.236965:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ba00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760793.236969:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454825d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27947008->27951103] (req 27947008->27951103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.237047:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454825d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27947008->27951103] (req 27947008->27951103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.237069:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454825d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27947008->27951103] (req 27947008->27951103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.237077:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d90d5862/0x493674835454825d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27947008->27951103] (req 27947008->27951103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.237266:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495957504:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36092us (36322us total) trans 4294974140 rc 0/0 00000001:00080000:0.0:1789760793.237297:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974140 is committed 00000100:00100000:1.0:1789760793.237310:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6852 00000100:00100000:1.0:1789760793.237320:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777494900222 time 777521875539 state 0x2 ref 0 00000100:00100000:3.0:1789760793.240770:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760793.240907:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495957632 00000100:00100000:1.0:1789760793.240917:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495957632 00000100:00100000:1.0:1789760793.240971:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777494900222 time 777521875539 state 0x0 ref 1 00000100:00100000:1.0:1789760793.241052:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777522135955 time 777521875539 state 0x1 ref 1 00000100:00100000:1.0:1789760793.241084:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777522135955 time 777561800999 state 0x0 ref 1 00000100:00100000:1.0:1789760793.241096:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495957632:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760793.241144:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27951104, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:1.0:1789760793.241153:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548264 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.241169:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=777s v=(0 0) 00010000:00010000:1.0:1789760793.241174:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.241180:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548264 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27951104->27955199] (req 27951104->27955199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.241193:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548264 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27951104->27955199] (req 27951104->27955199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.270781:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974141 is committed 00002000:00010000:1.0:1789760793.270887:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27951104 -> 27955200 00010000:00010000:1.0:1789760793.270902:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548264 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27951104->27955199] (req 27951104->27955199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.270915:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548264 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27951104->27955199] (req 27951104->27955199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.270931:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548264 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27951104->27955199] (req 27951104->27955199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.270939:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760793.270944:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548264 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27951104->27955199] (req 27951104->27955199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.270953:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548264 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27951104->27955199] (req 27951104->27955199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.270973:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548264 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27951104->27955199] (req 27951104->27955199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.270981:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548264 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27951104->27955199] (req 27951104->27955199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.271075:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034cdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495957632:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29976us (30309us total) trans 4294974141 rc 0/0 00000100:00100000:1.0:1789760793.271086:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6853 00000100:00100000:1.0:1789760793.271095:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777522135955 time 777561800999 state 0x2 ref 0 00000100:00100000:3.0:1789760793.274767:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760793.275231:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495957760 00000100:00100000:1.0:1789760793.275242:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495957760 00000100:00100000:1.0:1789760793.275335:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777522135955 time 777561800999 state 0x0 ref 1 00000100:00100000:1.0:1789760793.275349:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777562061415 time 777561800999 state 0x1 ref 1 00000100:00100000:1.0:1789760793.275373:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777562061415 time 777596090000 state 0x0 ref 1 00000100:00100000:1.0:1789760793.275620:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034ce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495957760:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760793.275683:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27955200, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:1.0:1789760793.275692:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454826b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.275709:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=777s v=(0 0) 00010000:00010000:1.0:1789760793.275713:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.275719:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454826b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27955200->27959295] (req 27955200->27959295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.275732:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454826b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27955200->27959295] (req 27955200->27959295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760793.314403:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27955200 -> 27959296 00010000:00010000:3.0:1789760793.314428:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454826b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27955200->27959295] (req 27955200->27959295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.314440:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454826b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27955200->27959295] (req 27955200->27959295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.314450:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454826b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27955200->27959295] (req 27955200->27959295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.314462:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760793.314468:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454826b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27955200->27959295] (req 27955200->27959295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.314478:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454826b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27955200->27959295] (req 27955200->27959295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.314587:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454826b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27955200->27959295] (req 27955200->27959295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.314596:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454826b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27955200->27959295] (req 27955200->27959295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.314697:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034ce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495957760:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39310us (39936us total) trans 4294974142 rc 0/0 00000100:00100000:3.0:1789760793.314708:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6854 00000100:00100000:3.0:1789760793.314718:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777562061415 time 777596090000 state 0x2 ref 0 00000001:00080000:3.0:1789760793.314754:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974142 is committed 00000100:00100000:3.0:1789760793.318415:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760793.318903:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495957888 00000100:00100000:1.0:1789760793.318913:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495957888 00000100:00100000:1.0:1789760793.318965:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777562061415 time 777596090000 state 0x0 ref 1 00000100:00100000:1.0:1789760793.318980:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777596350416 time 777596090000 state 0x1 ref 1 00000100:00100000:1.0:1789760793.319009:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777596350416 time 777639726291 state 0x0 ref 1 00000100:00100000:1.0:1789760793.319020:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f034ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495957888:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760793.319067:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27959296, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:1.0:1789760793.319076:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548272 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.319092:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=777s v=(0 0) 00010000:00010000:1.0:1789760793.319097:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.319104:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548272 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27959296->27963391] (req 27959296->27963391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.319118:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548272 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27959296->27963391] (req 27959296->27963391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.346194:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974143 is committed 00002000:00010000:1.0:1789760793.346346:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27959296 -> 27963392 00010000:00010000:1.0:1789760793.346361:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548272 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27959296->27963391] (req 27959296->27963391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.346373:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548272 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27959296->27963391] (req 27959296->27963391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.346382:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548272 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27959296->27963391] (req 27959296->27963391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.346389:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760793.346394:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548272 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27959296->27963391] (req 27959296->27963391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.346402:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548272 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27959296->27963391] (req 27959296->27963391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.346426:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548272 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27959296->27963391] (req 27959296->27963391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.346434:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548272 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27959296->27963391] (req 27959296->27963391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.346597:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f034ced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495957888:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27573us (28187us total) trans 4294974143 rc 0/0 00000100:00100000:1.0:1789760793.346607:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6855 00000100:00100000:1.0:1789760793.346616:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777596350416 time 777639726291 state 0x2 ref 0 00000100:00100000:2.0:1789760793.350648:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760793.350746:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495958016 00000100:00100000:1.0:1789760793.350756:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495958016 00000100:00100000:1.0:1789760793.350815:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777596350416 time 777639726291 state 0x0 ref 1 00000100:00100000:1.0:1789760793.350830:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777639986707 time 777639726291 state 0x1 ref 1 00000100:00100000:1.0:1789760793.350865:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777639986707 time 777671581062 state 0x0 ref 1 00000100:00100000:1.0:1789760793.350879:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495958016:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760793.350936:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27963392, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:1.0:1789760793.350945:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354548279 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.350961:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=777s v=(0 0) 00010000:00010000:1.0:1789760793.350966:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.350973:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354548279 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27963392->27967487] (req 27963392->27967487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.350986:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354548279 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27963392->27967487] (req 27963392->27967487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.382902:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974144 is committed 00002000:00010000:1.0:1789760793.383071:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27963392 -> 27967488 00010000:00010000:1.0:1789760793.383086:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354548279 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27963392->27967487] (req 27963392->27967487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.383099:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354548279 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27963392->27967487] (req 27963392->27967487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.383108:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354548279 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27963392->27967487] (req 27963392->27967487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.383114:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760793.383119:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354548279 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27963392->27967487] (req 27963392->27967487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.383127:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354548279 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27963392->27967487] (req 27963392->27967487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.383145:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354548279 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27963392->27967487] (req 27963392->27967487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.383152:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354548279 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27963392->27967487] (req 27963392->27967487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.383248:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495958016:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32367us (32605us total) trans 4294974144 rc 0/0 00000100:00100000:1.0:1789760793.383259:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6856 00000100:00100000:1.0:1789760793.383269:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777639986707 time 777671581062 state 0x2 ref 0 00000100:00100000:3.0:1789760793.387177:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760793.387439:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495958144 00000100:00100000:1.0:1789760793.387449:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495958144 00000100:00100000:1.0:1789760793.387616:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777639986707 time 777671581062 state 0x0 ref 1 00000100:00100000:1.0:1789760793.387631:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777671841478 time 777671581062 state 0x1 ref 1 00000100:00100000:1.0:1789760793.387702:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777671841478 time 777708418258 state 0x0 ref 1 00000100:00100000:1.0:1789760793.387715:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495958144:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760793.387766:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27967488, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:1.0:1789760793.387775:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354548280 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.387791:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=777s v=(0 0) 00010000:00010000:1.0:1789760793.387795:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.387801:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354548280 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27967488->27971583] (req 27967488->27971583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.387813:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354548280 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27967488->27971583] (req 27967488->27971583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760793.412071:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27967488 -> 27971584 00010000:00010000:1.0:1789760793.412091:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354548280 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27967488->27971583] (req 27967488->27971583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.412104:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354548280 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27967488->27971583] (req 27967488->27971583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.412113:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354548280 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27967488->27971583] (req 27967488->27971583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.412120:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760793.412125:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354548280 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27967488->27971583] (req 27967488->27971583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.412135:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354548280 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27967488->27971583] (req 27967488->27971583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.412156:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354548280 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27967488->27971583] (req 27967488->27971583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.412163:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x4936748354548280 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27967488->27971583] (req 27967488->27971583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.412257:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290113a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495958144:12345-192.168.203.9@tcp:4:dd.0 Request processed in 24540us (25084us total) trans 4294974145 rc 0/0 00000100:00100000:1.0:1789760793.412270:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6857 00000100:00100000:1.0:1789760793.412309:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777671841478 time 777708418258 state 0x2 ref 0 00000001:00080000:3.0:1789760793.412843:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974145 is committed 00000100:00100000:2.0:1789760793.415414:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760793.415823:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495958272 00000100:00100000:1.0:1789760793.415834:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495958272 00000100:00100000:1.0:1789760793.415890:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777671841478 time 777708418258 state 0x0 ref 1 00000100:00100000:1.0:1789760793.415904:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777708678674 time 777708418258 state 0x1 ref 1 00000100:00100000:1.0:1789760793.415926:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777708678674 time 777736643392 state 0x0 ref 1 00000100:00100000:1.0:1789760793.415937:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495958272:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760793.415985:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27971584, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:1.0:1789760793.415994:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548287 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.416009:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=777s v=(0 0) 00010000:00010000:1.0:1789760793.416013:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.416020:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548287 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27971584->27975679] (req 27971584->27975679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.416033:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548287 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27971584->27975679] (req 27971584->27975679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.438561:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760793.457330:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974146 is committed 00002000:00010000:3.0:1789760793.457397:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27971584 -> 27975680 00010000:00010000:3.0:1789760793.457412:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548287 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27971584->27975679] (req 27971584->27975679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.457426:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548287 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27971584->27975679] (req 27971584->27975679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.457434:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548287 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27971584->27975679] (req 27971584->27975679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.457441:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760793.457445:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548287 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27971584->27975679] (req 27971584->27975679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.457454:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548287 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27971584->27975679] (req 27971584->27975679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.457472:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548287 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27971584->27975679] (req 27971584->27975679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.457479:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548287 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27971584->27975679] (req 27971584->27975679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.457574:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495958272:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41633us (42164us total) trans 4294974146 rc 0/0 00000100:00100000:3.0:1789760793.457586:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6858 00000100:00100000:3.0:1789760793.457595:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777708678674 time 777736643392 state 0x2 ref 0 00000100:00100000:2.0:1789760793.461231:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760793.461750:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495958400 00000100:00100000:1.0:1789760793.461760:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495958400 00000100:00100000:1.0:1789760793.461817:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777708678674 time 777736643392 state 0x0 ref 1 00000100:00100000:1.0:1789760793.461833:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777736903808 time 777736643392 state 0x1 ref 1 00000100:00100000:1.0:1789760793.461868:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777736903808 time 777782584949 state 0x0 ref 1 00000100:00100000:1.0:1789760793.461880:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495958400:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760793.461932:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27975680, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:1.0:1789760793.461941:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454828e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.461959:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=777s v=(0 0) 00010000:00010000:1.0:1789760793.461964:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.461970:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454828e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27975680->27979775] (req 27975680->27979775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.462114:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454828e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27975680->27979775] (req 27975680->27979775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760793.495061:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27975680 -> 27979776 00000001:00080000:3.0:1789760793.495079:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974147 is committed 00010000:00010000:2.0:1789760793.495083:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454828e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27975680->27979775] (req 27975680->27979775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.495095:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454828e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27975680->27979775] (req 27975680->27979775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.495105:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454828e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27975680->27979775] (req 27975680->27979775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.495113:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760793.495118:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454828e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27975680->27979775] (req 27975680->27979775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.495127:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454828e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27975680->27979775] (req 27975680->27979775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.495152:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454828e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27975680->27979775] (req 27975680->27979775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.495160:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x493674835454828e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27975680->27979775] (req 27975680->27979775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.495266:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495958400:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33382us (34038us total) trans 4294974147 rc 0/0 00000100:00100000:2.0:1789760793.495277:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6859 00000100:00100000:2.0:1789760793.495326:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777736903808 time 777782584949 state 0x2 ref 0 00000100:00100000:1.0:1789760793.502727:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760793.502815:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495958528 00000100:00100000:3.0:1789760793.502826:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495958528 00000100:00100000:3.0:1789760793.502884:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777736903808 time 777782584949 state 0x0 ref 1 00000100:00100000:3.0:1789760793.502899:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777782845365 time 777782584949 state 0x1 ref 1 00000100:00100000:3.0:1789760793.502931:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777782845365 time 777823648403 state 0x0 ref 1 00000100:00100000:3.0:1789760793.502943:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036e57100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495958528:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760793.502990:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27979776, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:3.0:1789760793.503000:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354548295 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.503016:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760793.503021:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.503027:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354548295 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27979776->27983871] (req 27979776->27983871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.503043:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354548295 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27979776->27983871] (req 27979776->27983871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.559095:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974148 is committed 00002000:00010000:1.0:1789760793.559349:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27979776 -> 27983872 00010000:00010000:1.0:1789760793.559364:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354548295 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27979776->27983871] (req 27979776->27983871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.559375:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354548295 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27979776->27983871] (req 27979776->27983871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.559389:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354548295 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27979776->27983871] (req 27979776->27983871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.559396:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760793.559401:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354548295 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27979776->27983871] (req 27979776->27983871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.559409:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354548295 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27979776->27983871] (req 27979776->27983871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.559427:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354548295 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27979776->27983871] (req 27979776->27983871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.559435:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354548295 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27979776->27983871] (req 27979776->27983871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.559528:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036e57100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495958528:12345-192.168.203.9@tcp:4:dd.0 Request processed in 56581us (56806us total) trans 4294974148 rc 0/0 00000100:00100000:1.0:1789760793.559538:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6860 00000100:00100000:1.0:1789760793.559548:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777782845365 time 777823648403 state 0x2 ref 0 00000100:00100000:3.0:1789760793.566568:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760793.566780:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495958656 00000100:00100000:0.0:1789760793.566789:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495958656 00000100:00100000:0.0:1789760793.566847:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777782845365 time 777823648403 state 0x0 ref 1 00000100:00100000:0.0:1789760793.566862:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777823908819 time 777823648403 state 0x1 ref 1 00000100:00100000:0.0:1789760793.566893:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777823908819 time 777887609929 state 0x0 ref 1 00000100:00100000:0.0:1789760793.566906:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495958656:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760793.566951:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27983872, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:0.0:1789760793.566960:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454829c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.566987:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=777s v=(0 0) 00010000:00010000:0.0:1789760793.566992:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.566999:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454829c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27983872->27987967] (req 27983872->27987967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.567012:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454829c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27983872->27987967] (req 27983872->27987967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.603259:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974149 is committed 00002000:00010000:0.0:1789760793.603328:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27983872 -> 27987968 00010000:00010000:0.0:1789760793.603345:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454829c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27983872->27987967] (req 27983872->27987967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.603364:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454829c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27983872->27987967] (req 27983872->27987967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.603374:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454829c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27983872->27987967] (req 27983872->27987967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.603382:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760793.603387:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454829c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27983872->27987967] (req 27983872->27987967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.603396:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454829c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27983872->27987967] (req 27983872->27987967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.603416:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454829c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27983872->27987967] (req 27983872->27987967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.603424:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x493674835454829c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27983872->27987967] (req 27983872->27987967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.603507:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495958656:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36600us (36943us total) trans 4294974149 rc 0/0 00000100:00100000:0.0:1789760793.603516:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6861 00000100:00100000:0.0:1789760793.603525:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777823908819 time 777887609929 state 0x2 ref 0 00000100:00100000:3.0:1789760793.607538:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760793.607889:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495958784 00000100:00100000:2.0:1789760793.607899:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495958784 00000100:00100000:2.0:1789760793.607948:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777823908819 time 777887609929 state 0x0 ref 1 00000100:00100000:2.0:1789760793.607960:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777887870345 time 777887609929 state 0x1 ref 1 00000100:00100000:2.0:1789760793.607989:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777887870345 time 777928705955 state 0x0 ref 1 00000100:00100000:2.0:1789760793.608000:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495958784:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760793.608045:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27987968, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:2.0:1789760793.608054:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545482a3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.608074:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=777s v=(0 0) 00010000:00010000:2.0:1789760793.608079:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.608085:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545482a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27987968->27992063] (req 27987968->27992063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.608096:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545482a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27987968->27992063] (req 27987968->27992063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760793.645893:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27987968 -> 27992064 00010000:00010000:3.0:1789760793.645911:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545482a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27987968->27992063] (req 27987968->27992063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.645924:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545482a3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27987968->27992063] (req 27987968->27992063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.645932:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545482a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27987968->27992063] (req 27987968->27992063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.645939:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760793.645944:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545482a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27987968->27992063] (req 27987968->27992063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.645953:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545482a3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27987968->27992063] (req 27987968->27992063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.645972:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545482a3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27987968->27992063] (req 27987968->27992063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.645979:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545482a3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27987968->27992063] (req 27987968->27992063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.646065:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495958784:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38061us (38532us total) trans 4294974150 rc 0/0 00000100:00100000:3.0:1789760793.646075:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6862 00000100:00100000:3.0:1789760793.646085:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777887870345 time 777928705955 state 0x2 ref 0 00000001:00080000:3.0:1789760793.646126:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974150 is committed 00000100:00100000:1.0:1789760793.651438:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760793.651523:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495958912 00000100:00100000:3.0:1789760793.651533:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495958912 00000100:00100000:3.0:1789760793.651586:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777887870345 time 777928705955 state 0x0 ref 1 00000100:00100000:3.0:1789760793.651601:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777928966371 time 777928705955 state 0x1 ref 1 00000100:00100000:3.0:1789760793.651641:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777928966371 time 777972349007 state 0x0 ref 1 00000100:00100000:3.0:1789760793.651662:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495958912:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760793.651708:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27992064, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:3.0:1789760793.651718:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545482aa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.651733:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760793.651738:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.651743:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545482aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27992064->27996159] (req 27992064->27996159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.651757:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545482aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27992064->27996159] (req 27992064->27996159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760793.677694:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27992064 -> 27996160 00002000:00010000:0.0:1789760793.677703:0:13059:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760792 -> 1789760793 00002000:00010000:0.0:1789760793.677706:0:13059:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760792 -> 1789760793 00010000:00010000:0.0:1789760793.677718:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545482aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27992064->27996159] (req 27992064->27996159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.677731:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545482aa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27992064->27996159] (req 27992064->27996159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.677740:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545482aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27992064->27996159] (req 27992064->27996159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.677748:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760793.677754:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545482aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27992064->27996159] (req 27992064->27996159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.677763:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545482aa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27992064->27996159] (req 27992064->27996159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.677781:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545482aa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27992064->27996159] (req 27992064->27996159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.677790:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545482aa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27992064->27996159] (req 27992064->27996159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.677985:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495958912:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26329us (26551us total) trans 4294974151 rc 0/0 00000100:00100000:0.0:1789760793.677995:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6863 00000100:00100000:0.0:1789760793.678006:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777928966371 time 777972349007 state 0x2 ref 0 00000001:00080000:2.0:1789760793.679000:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974151 is committed 00000100:00100000:2.0:1789760793.680598:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760793.680681:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495959040 00000100:00100000:0.0:1789760793.680691:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495959040 00000100:00100000:0.0:1789760793.680742:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777928966371 time 777972349007 state 0x0 ref 1 00000100:00100000:0.0:1789760793.680758:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777972609423 time 777972349007 state 0x1 ref 1 00000100:00100000:0.0:1789760793.680790:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777972609423 time 778001506188 state 0x0 ref 1 00000100:00100000:0.0:1789760793.680802:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495959040:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760793.680855:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27996160, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd809 00010000:00010000:0.0:1789760793.680864:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545482b1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.680879:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=778s v=(0 0) 00010000:00010000:0.0:1789760793.680883:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.680889:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545482b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27996160->28000255] (req 27996160->28000255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.680905:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545482b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27996160->28000255] (req 27996160->28000255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.718975:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974152 is committed 00002000:00010000:1.0:1789760793.719151:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27996160 -> 28000256 00010000:00010000:1.0:1789760793.719167:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545482b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27996160->28000255] (req 27996160->28000255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.719185:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545482b1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27996160->28000255] (req 27996160->28000255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.719193:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545482b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27996160->28000255] (req 27996160->28000255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.719200:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760793.719205:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545482b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27996160->28000255] (req 27996160->28000255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.719215:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545482b1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27996160->28000255] (req 27996160->28000255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.719234:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545482b1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27996160->28000255] (req 27996160->28000255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.719241:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x49367483545482b1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27996160->28000255] (req 27996160->28000255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.719374:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495959040:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38569us (38778us total) trans 4294974152 rc 0/0 00000100:00100000:1.0:1789760793.719385:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6864 00000100:00100000:1.0:1789760793.719395:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777972609423 time 778001506188 state 0x2 ref 0 00000100:00100000:3.0:1789760793.723769:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760793.723857:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495959168 00000100:00100000:2.0:1789760793.723868:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495959168 00000100:00100000:2.0:1789760793.723923:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 777972609423 time 778001506188 state 0x0 ref 1 00000100:00100000:2.0:1789760793.723937:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778001766604 time 778001506188 state 0x1 ref 1 00000100:00100000:2.0:1789760793.723967:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778001766604 time 778044684248 state 0x0 ref 1 00000100:00100000:2.0:1789760793.723979:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fba2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495959168:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760793.724022:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28000256, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd809 00010000:00010000:2.0:1789760793.724031:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545482b8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.724047:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=778s v=(0 0) 00010000:00010000:2.0:1789760793.724052:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.724059:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545482b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28000256->28004351] (req 28000256->28004351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.724078:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545482b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28000256->28004351] (req 28000256->28004351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.764105:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974153 is committed 00002000:00010000:2.0:1789760793.765259:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28000256 -> 28004352 00010000:00010000:2.0:1789760793.765311:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545482b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28000256->28004351] (req 28000256->28004351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.765325:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545482b8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28000256->28004351] (req 28000256->28004351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.765334:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545482b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28000256->28004351] (req 28000256->28004351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.765341:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760793.765346:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545482b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28000256->28004351] (req 28000256->28004351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.765355:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545482b8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28000256->28004351] (req 28000256->28004351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.765375:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545482b8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28000256->28004351] (req 28000256->28004351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.765383:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545482b8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28000256->28004351] (req 28000256->28004351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.765482:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fba2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495959168:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41500us (41717us total) trans 4294974153 rc 0/0 00000100:00100000:2.0:1789760793.765492:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6865 00000100:00100000:2.0:1789760793.765503:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778001766604 time 778044684248 state 0x2 ref 0 00000100:00100000:0.0:1789760793.770391:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760793.771079:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495959296 00000100:00100000:3.0:1789760793.771089:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495959296 00000100:00100000:3.0:1789760793.771149:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778001766604 time 778044684248 state 0x0 ref 1 00000100:00100000:3.0:1789760793.771164:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778044944664 time 778044684248 state 0x1 ref 1 00000100:00100000:3.0:1789760793.771196:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778044944664 time 778091912664 state 0x0 ref 1 00000100:00100000:3.0:1789760793.771208:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01374a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495959296:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760793.771257:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28004352, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd809 00010000:00010000:3.0:1789760793.771267:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482bf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.771320:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=778s v=(0 0) 00010000:00010000:3.0:1789760793.771325:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.771333:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28004352->28008447] (req 28004352->28008447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.771347:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28004352->28008447] (req 28004352->28008447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.796735:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974154 is committed 00002000:00010000:3.0:1789760793.796850:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28004352 -> 28008448 00010000:00010000:3.0:1789760793.796867:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28004352->28008447] (req 28004352->28008447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.796881:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482bf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28004352->28008447] (req 28004352->28008447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.796890:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28004352->28008447] (req 28004352->28008447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.796898:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760793.796904:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28004352->28008447] (req 28004352->28008447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.796913:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482bf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28004352->28008447] (req 28004352->28008447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.796933:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482bf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28004352->28008447] (req 28004352->28008447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.796940:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482bf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28004352->28008447] (req 28004352->28008447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.797154:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01374a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495959296:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25942us (26771us total) trans 4294974154 rc 0/0 00000100:00100000:3.0:1789760793.797166:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6866 00000100:00100000:3.0:1789760793.797175:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778044944664 time 778091912664 state 0x2 ref 0 00000100:00100000:2.0:1789760793.801778:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760793.801890:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495959424 00000100:00100000:1.0:1789760793.801900:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495959424 00000100:00100000:1.0:1789760793.801950:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778044944664 time 778091912664 state 0x0 ref 1 00000100:00100000:1.0:1789760793.801966:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778092173080 time 778091912664 state 0x1 ref 1 00000100:00100000:1.0:1789760793.802084:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778092173080 time 778122799522 state 0x0 ref 1 00000100:00100000:1.0:1789760793.802098:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fdd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495959424:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760793.802144:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28008448, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd809 00010000:00010000:1.0:1789760793.802152:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545482c6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.802168:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760793.802172:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.802178:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545482c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28008448->28012543] (req 28008448->28012543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.802193:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545482c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28008448->28012543] (req 28008448->28012543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.831344:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974155 is committed 00002000:00010000:0.0:1789760793.831447:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28008448 -> 28012544 00010000:00010000:0.0:1789760793.831463:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545482c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28008448->28012543] (req 28008448->28012543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.831477:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545482c6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28008448->28012543] (req 28008448->28012543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.831486:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545482c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28008448->28012543] (req 28008448->28012543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.831493:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760793.831498:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545482c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28008448->28012543] (req 28008448->28012543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.831507:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545482c6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28008448->28012543] (req 28008448->28012543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.831530:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545482c6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28008448->28012543] (req 28008448->28012543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.831538:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545482c6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28008448->28012543] (req 28008448->28012543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.832046:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fdd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495959424:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29945us (30272us total) trans 4294974155 rc 0/0 00000100:00100000:0.0:1789760793.832058:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6867 00000100:00100000:0.0:1789760793.832068:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778092173080 time 778122799522 state 0x2 ref 0 00000100:00100000:2.0:1789760793.835272:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760793.835752:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495959552 00000100:00100000:0.0:1789760793.835762:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495959552 00000100:00100000:0.0:1789760793.835813:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778092173080 time 778122799522 state 0x0 ref 1 00000100:00100000:0.0:1789760793.835829:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778123059938 time 778122799522 state 0x1 ref 1 00000100:00100000:0.0:1789760793.835898:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778123059938 time 778156614185 state 0x0 ref 1 00000100:00100000:0.0:1789760793.835911:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495959552:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760793.835958:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28012544, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd809 00010000:00010000:0.0:1789760793.835966:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545482cd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.835991:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=778s v=(0 0) 00010000:00010000:0.0:1789760793.835995:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.836002:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545482cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28012544->28016639] (req 28012544->28016639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.836016:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545482cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28012544->28016639] (req 28012544->28016639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760793.864259:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28012544 -> 28016640 00010000:00010000:2.0:1789760793.864307:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545482cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28012544->28016639] (req 28012544->28016639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.864320:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545482cd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28012544->28016639] (req 28012544->28016639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.864329:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545482cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28012544->28016639] (req 28012544->28016639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.864337:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760793.864342:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545482cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28012544->28016639] (req 28012544->28016639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.864350:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545482cd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28012544->28016639] (req 28012544->28016639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.864374:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545482cd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28012544->28016639] (req 28012544->28016639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.864382:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545482cd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28012544->28016639] (req 28012544->28016639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.864466:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fde300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495959552:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28553us (29197us total) trans 4294974156 rc 0/0 00000100:00100000:2.0:1789760793.864476:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6868 00000100:00100000:2.0:1789760793.864486:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778123059938 time 778156614185 state 0x2 ref 0 00000001:00080000:3.0:1789760793.865870:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974156 is committed 00000100:00100000:1.0:1789760793.868082:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760793.868163:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495959680 00000100:00100000:3.0:1789760793.868172:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495959680 00000100:00100000:3.0:1789760793.868224:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778123059938 time 778156614185 state 0x0 ref 1 00000100:00100000:3.0:1789760793.868239:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778156874601 time 778156614185 state 0x1 ref 1 00000100:00100000:3.0:1789760793.868266:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778156874601 time 778188982748 state 0x0 ref 1 00000100:00100000:3.0:1789760793.868314:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495959680:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760793.868366:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28016640, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd809 00010000:00010000:3.0:1789760793.868375:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482d4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.868390:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=778s v=(0 0) 00010000:00010000:3.0:1789760793.868394:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.868401:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28016640->28020735] (req 28016640->28020735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.868414:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28016640->28020735] (req 28016640->28020735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.889926:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f01375180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:59c4e264-9b75-482b-8813-3d1dc124f384:3317:1876699520317312:0@lo:400:kworker.0 00000100:00100000:0.0:1789760793.890011:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760793.890062:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f01376a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3317:1876699520317824:0@lo:400:kworker.0 00000100:00100000:0.0:1789760793.890089:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760793.890113:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1789760793.890119:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f01376300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3317:1876699520317440:0@lo:400:kworker.0 00000100:00100000:0.0:1789760793.890140:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760793.890157:0:3317:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f01377100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3317:1876699520317952:0@lo:400:kworker.0 00000100:00100000:0.0:1789760793.890179:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760793.890208:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520317312 00000100:00100000:0.0:1789760793.890214:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520317312 00000100:00100000:0.0:1789760793.890253:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 421 00000100:00100000:0.0:1789760793.890261:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290122ccc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:59c4e264-9b75-482b-8813-3d1dc124f384+19:3317:x1876699520317312:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760793.890331:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290122ccc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:59c4e264-9b75-482b-8813-3d1dc124f384+19:3317:x1876699520317312:12345-0@lo:400:kworker.0 Request processed in 69us (319us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760793.890337:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 421 00000100:00100000:0.0:1789760793.890367:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520317952 00000100:00100000:0.0:1789760793.890371:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520317952 00000100:00100000:0.0:1789760793.891097:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 244 00000100:00100000:0.0:1789760793.891108:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01375f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3317:x1876699520317952:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760793.891147:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01375f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3317:x1876699520317952:12345-0@lo:400:kworker.0 Request processed in 42us (967us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760793.891153:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 244 00000100:00080000:0.0:1789760793.891173:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789760793.891202:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f01377100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3317:1876699520317952:0@lo:400:kworker.0 00000100:00100000:0.0:1789760793.891213:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f01375180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:59c4e264-9b75-482b-8813-3d1dc124f384:3317:1876699520317312:0@lo:400:kworker.0 00000100:00100000:3.0:1789760793.891320:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520317824 00000100:00100000:3.0:1789760793.891326:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520317824 00000100:00100000:3.0:1789760793.891354:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520317440 00000100:00100000:3.0:1789760793.891357:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520317440 00000100:00100000:3.0:1789760793.891373:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 532 00000100:00100000:3.0:1789760793.891380:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01375880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1876699520317824:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760793.891416:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01375880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1876699520317824:12345-0@lo:400:kworker.0 Request processed in 36us (1326us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760793.891422:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 532 00000100:00100000:3.0:1789760793.891434:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 533 00000100:00100000:3.0:1789760793.891439:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01377b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1876699520317440:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760793.891464:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01377b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1876699520317440:12345-0@lo:400:kworker.0 Request processed in 25us (1323us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760793.891470:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 533 00000100:00080000:0.0:1789760793.891883:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760793.891905:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f01376300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3317:1876699520317440:0@lo:400:kworker.0 00000100:00100000:0.0:1789760793.891916:0:3317:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f01376a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3317:1876699520317824:0@lo:400:kworker.0 00000100:00100000:1.0:1789760793.893123:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f01377800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520317696:0@lo:400:kworker.0 00000100:00100000:1.0:1789760793.893175:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760793.893205:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789760793.893213:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff928f01374e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699520317568:0@lo:400:kworker.0 00000100:00100000:1.0:1789760793.893235:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760793.893270:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520317696 00000100:00100000:1.0:1789760793.893277:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520317696 00000100:00100000:1.0:1789760793.893339:0:6709:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 245 00000100:00100000:1.0:1789760793.893347:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520317696:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760793.893391:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520317696:12345-0@lo:400:kworker.0 Request processed in 45us (215us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760793.893398:0:6709:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 245 00000100:00100000:1.0:1789760793.893419:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520317568 00000100:00100000:1.0:1789760793.893423:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520317568 00000100:00100000:1.0:1789760793.893447:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 534 00000100:00100000:1.0:1789760793.893453:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699520317568:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760793.893482:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f65880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699520317568:12345-0@lo:400:kworker.0 Request processed in 29us (246us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760793.893487:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 534 00000100:00080000:1.0:1789760793.893509:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789760793.893513:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760793.893532:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f01374e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699520317568:0@lo:400:kworker.0 00000100:00100000:1.0:1789760793.893542:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff928f01377800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520317696:0@lo:400:kworker.0 00000001:00080000:1.0:1789760793.899298:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974157 is committed 00002000:00010000:3.0:1789760793.899740:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28016640 -> 28020736 00010000:00010000:3.0:1789760793.899755:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28016640->28020735] (req 28016640->28020735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.899769:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482d4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28016640->28020735] (req 28016640->28020735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.899778:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28016640->28020735] (req 28016640->28020735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.899786:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760793.899791:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28016640->28020735] (req 28016640->28020735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.899801:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482d4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28016640->28020735] (req 28016640->28020735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.899821:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482d4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28016640->28020735] (req 28016640->28020735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.899829:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482d4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28016640->28020735] (req 28016640->28020735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.899956:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495959680:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31673us (31876us total) trans 4294974157 rc 0/0 00000100:00100000:3.0:1789760793.899966:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6869 00000100:00100000:3.0:1789760793.900038:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778156874601 time 778188982748 state 0x2 ref 0 00000100:00100000:1.0:1789760793.903225:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760793.903327:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495959808 00000100:00100000:0.0:1789760793.903337:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495959808 00000100:00100000:0.0:1789760793.903389:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778156874601 time 778188982748 state 0x0 ref 1 00000100:00100000:0.0:1789760793.903404:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778189243164 time 778188982748 state 0x1 ref 1 00000100:00100000:0.0:1789760793.903432:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778189243164 time 778224148702 state 0x0 ref 1 00000100:00100000:0.0:1789760793.903444:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495959808:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760793.903492:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28020736, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd809 00010000:00010000:0.0:1789760793.903501:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545482db lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.903516:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=778s v=(0 0) 00010000:00010000:0.0:1789760793.903521:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.903527:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545482db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28020736->28024831] (req 28020736->28024831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.903540:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545482db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28020736->28024831] (req 28020736->28024831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.926127:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974158 is committed 00002000:00010000:1.0:1789760793.926238:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28020736 -> 28024832 00010000:00010000:1.0:1789760793.926253:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545482db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28020736->28024831] (req 28020736->28024831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.926265:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545482db lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28020736->28024831] (req 28020736->28024831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.926298:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545482db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28020736->28024831] (req 28020736->28024831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.926307:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760793.926312:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545482db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28020736->28024831] (req 28020736->28024831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.926320:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545482db lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28020736->28024831] (req 28020736->28024831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.926339:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545482db lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28020736->28024831] (req 28020736->28024831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.926347:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x49367483545482db lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28020736->28024831] (req 28020736->28024831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.926470:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495959808:12345-192.168.203.9@tcp:4:dd.0 Request processed in 23022us (23249us total) trans 4294974158 rc 0/0 00000100:00100000:1.0:1789760793.926480:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6870 00000100:00100000:1.0:1789760793.926489:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778189243164 time 778224148702 state 0x2 ref 0 00000100:00100000:2.0:1789760793.929556:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760793.929667:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495959936 00000100:00100000:0.0:1789760793.929676:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495959936 00000100:00100000:0.0:1789760793.929719:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778189243164 time 778224148702 state 0x0 ref 1 00000100:00100000:0.0:1789760793.929732:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778224409118 time 778224148702 state 0x1 ref 1 00000100:00100000:0.0:1789760793.929799:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778224409118 time 778250515395 state 0x0 ref 1 00000100:00100000:0.0:1789760793.929811:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495959936:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760793.929856:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28024832, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd809 00010000:00010000:0.0:1789760793.929864:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545482e2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.929879:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=778s v=(0 0) 00010000:00010000:0.0:1789760793.929884:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.929890:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545482e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28024832->28028927] (req 28024832->28028927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.929903:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545482e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28024832->28028927] (req 28024832->28028927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.956176:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974159 is committed 00002000:00010000:3.0:1789760793.956297:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28024832 -> 28028928 00010000:00010000:3.0:1789760793.956309:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545482e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28024832->28028927] (req 28024832->28028927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.956318:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545482e2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28024832->28028927] (req 28024832->28028927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.956324:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545482e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28024832->28028927] (req 28024832->28028927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.956330:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ae00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760793.956334:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545482e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28024832->28028927] (req 28024832->28028927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.956339:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545482e2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28024832->28028927] (req 28024832->28028927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.956354:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545482e2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28024832->28028927] (req 28024832->28028927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.956359:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x49367483545482e2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28024832->28028927] (req 28024832->28028927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.956445:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495959936:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26631us (26892us total) trans 4294974159 rc 0/0 00000100:00100000:3.0:1789760793.956455:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6871 00000100:00100000:3.0:1789760793.956464:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778224409118 time 778250515395 state 0x2 ref 0 00000100:00100000:0.0:1789760793.959155:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760793.959234:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495960064 00000100:00100000:3.0:1789760793.959243:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495960064 00000100:00100000:3.0:1789760793.959323:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778224409118 time 778250515395 state 0x0 ref 1 00000100:00100000:3.0:1789760793.959338:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778250775811 time 778250515395 state 0x1 ref 1 00000100:00100000:3.0:1789760793.959364:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778250775811 time 778280081208 state 0x0 ref 1 00000100:00100000:3.0:1789760793.959380:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01376a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495960064:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760793.959426:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28028928, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd809 00010000:00010000:3.0:1789760793.959435:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482e9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.959451:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=778s v=(0 0) 00010000:00010000:3.0:1789760793.959456:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.959463:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28028928->28033023] (req 28028928->28033023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.959477:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28028928->28033023] (req 28028928->28033023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.986857:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974160 is committed 00002000:00010000:1.0:1789760793.988375:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28028928 -> 28033024 00010000:00010000:1.0:1789760793.988395:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28028928->28033023] (req 28028928->28033023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.988414:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482e9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28028928->28033023] (req 28028928->28033023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.988423:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28028928->28033023] (req 28028928->28033023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.988438:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760793.988444:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28028928->28033023] (req 28028928->28033023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.988454:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482e9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28028928->28033023] (req 28028928->28033023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.988473:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482e9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28028928->28033023] (req 28028928->28033023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.988480:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545482e9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28028928->28033023] (req 28028928->28033023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.988629:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01376a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495960064:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29245us (29479us total) trans 4294974160 rc 0/0 00000100:00100000:1.0:1789760793.988640:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6872 00000100:00100000:1.0:1789760793.988659:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778250775811 time 778280081208 state 0x2 ref 0 00000100:00100000:0.0:1789760793.998202:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760793.998771:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495960192 00000100:00100000:1.0:1789760793.998781:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495960192 00000100:00100000:1.0:1789760793.998834:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778250775811 time 778280081208 state 0x0 ref 1 00000100:00100000:1.0:1789760793.998849:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778280341624 time 778280081208 state 0x1 ref 1 00000100:00100000:1.0:1789760793.998881:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778280341624 time 778319597425 state 0x0 ref 1 00000100:00100000:1.0:1789760793.998893:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01376300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495960192:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760793.998937:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28033024, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd809 00010000:00010000:1.0:1789760793.998946:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545482f0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.998962:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760793.998966:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.998973:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545482f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28033024->28037119] (req 28033024->28037119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.998986:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545482f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28033024->28037119] (req 28033024->28037119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.048791:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974161 is committed 00002000:00010000:1.0:1789760794.049269:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28033024 -> 28037120 00010000:00010000:1.0:1789760794.049476:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545482f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28033024->28037119] (req 28033024->28037119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.049490:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545482f0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28033024->28037119] (req 28033024->28037119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.049504:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545482f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28033024->28037119] (req 28033024->28037119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.049511:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760794.049516:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545482f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28033024->28037119] (req 28033024->28037119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.049525:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545482f0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28033024->28037119] (req 28033024->28037119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.049544:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545482f0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28033024->28037119] (req 28033024->28037119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.049552:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008fb0fa7d/0x49367483545482f0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28033024->28037119] (req 28033024->28037119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.049731:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01376300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495960192:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50833us (51531us total) trans 4294974161 rc 0/0 00000100:00100000:2.0:1789760794.049741:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6873 00000100:00100000:2.0:1789760794.049750:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778280341624 time 778319597425 state 0x2 ref 0 00000100:00100000:2.0:1789760794.058093:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760794.058194:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495960320 00000100:00100000:3.0:1789760794.058204:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495960320 00000100:00100000:3.0:1789760794.058258:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778280341624 time 778319597425 state 0x0 ref 1 00000100:00100000:3.0:1789760794.058273:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778319857841 time 778319597425 state 0x1 ref 1 00000100:00100000:3.0:1789760794.058343:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778319857841 time 778379058386 state 0x0 ref 1 00000100:00100000:3.0:1789760794.058355:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fdf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495960320:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760794.058398:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28037120, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd809 00010000:00010000:3.0:1789760794.058407:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545482f7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.058429:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=778s v=(0 0) 00010000:00010000:3.0:1789760794.058433:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.058439:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545482f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28037120->28041215] (req 28037120->28041215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.058453:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545482f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28037120->28041215] (req 28037120->28041215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760794.091474:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28037120 -> 28041216 00010000:00010000:2.0:1789760794.091494:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545482f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28037120->28041215] (req 28037120->28041215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.091507:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545482f7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28037120->28041215] (req 28037120->28041215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.091516:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545482f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28037120->28041215] (req 28037120->28041215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.091524:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760794.091529:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545482f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28037120->28041215] (req 28037120->28041215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.091541:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545482f7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28037120->28041215] (req 28037120->28041215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.091561:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545482f7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28037120->28041215] (req 28037120->28041215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.091570:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x49367483545482f7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28037120->28041215] (req 28037120->28041215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.091679:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fdf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495960320:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33321us (33601us total) trans 4294974162 rc 0/0 00000100:00100000:2.0:1789760794.091689:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6874 00000100:00100000:2.0:1789760794.091700:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778319857841 time 778379058386 state 0x2 ref 0 00000001:00080000:2.0:1789760794.091747:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974162 is committed 00000100:00100000:0.0:1789760794.100779:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760794.100865:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495960448 00000100:00100000:1.0:1789760794.100874:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495960448 00000100:00100000:1.0:1789760794.100921:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778319857841 time 778379058386 state 0x0 ref 1 00000100:00100000:1.0:1789760794.100935:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778379318802 time 778379058386 state 0x1 ref 1 00000100:00100000:1.0:1789760794.100988:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778379318802 time 778421705270 state 0x0 ref 1 00000100:00100000:1.0:1789760794.100999:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01375180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495960448:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760794.101043:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28041216, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd809 00010000:00010000:1.0:1789760794.101052:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545482fe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.101067:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760794.101071:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.101077:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545482fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28041216->28045311] (req 28041216->28045311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.101090:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545482fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28041216->28045311] (req 28041216->28045311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.132116:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974163 is committed 00002000:00010000:1.0:1789760794.132220:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28041216 -> 28045312 00010000:00010000:1.0:1789760794.132235:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545482fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28041216->28045311] (req 28041216->28045311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.132248:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545482fe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28041216->28045311] (req 28041216->28045311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.132257:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545482fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28041216->28045311] (req 28041216->28045311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.132264:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760794.132269:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545482fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28041216->28045311] (req 28041216->28045311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.132305:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545482fe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28041216->28045311] (req 28041216->28045311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.132326:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545482fe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28041216->28045311] (req 28041216->28045311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.132334:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545482fe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28041216->28045311] (req 28041216->28045311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.132414:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01375180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495960448:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31411us (31638us total) trans 4294974163 rc 0/0 00000100:00100000:1.0:1789760794.132424:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6875 00000100:00100000:1.0:1789760794.132434:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778379318802 time 778421705270 state 0x2 ref 0 00000100:00100000:1.0:1789760794.141606:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760794.141714:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495960576 00000100:00100000:2.0:1789760794.141725:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495960576 00000100:00100000:2.0:1789760794.141778:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778379318802 time 778421705270 state 0x0 ref 1 00000100:00100000:2.0:1789760794.141793:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778421965686 time 778421705270 state 0x1 ref 1 00000100:00100000:2.0:1789760794.141860:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778421965686 time 778462576745 state 0x0 ref 1 00000100:00100000:2.0:1789760794.141872:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495960576:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760794.141920:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28045312, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd809 00010000:00010000:2.0:1789760794.141930:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548305 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.141946:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=778s v=(0 0) 00010000:00010000:2.0:1789760794.141950:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.141957:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548305 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28045312->28049407] (req 28045312->28049407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.141971:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548305 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28045312->28049407] (req 28045312->28049407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.187455:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974164 is committed 00002000:00010000:1.0:1789760794.187829:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28045312 -> 28049408 00010000:00010000:1.0:1789760794.187846:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548305 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28045312->28049407] (req 28045312->28049407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.187858:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548305 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28045312->28049407] (req 28045312->28049407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.187867:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548305 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28045312->28049407] (req 28045312->28049407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.187875:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760794.187880:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548305 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28045312->28049407] (req 28045312->28049407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.187906:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548305 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28045312->28049407] (req 28045312->28049407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.187925:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548305 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28045312->28049407] (req 28045312->28049407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.187933:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548305 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28045312->28049407] (req 28045312->28049407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.188019:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495960576:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46145us (46419us total) trans 4294974164 rc 0/0 00000100:00100000:1.0:1789760794.188028:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6876 00000100:00100000:1.0:1789760794.188037:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778421965686 time 778462576745 state 0x2 ref 0 00000100:00100000:2.0:1789760794.192054:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760794.192148:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495960704 00000100:00100000:3.0:1789760794.192159:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495960704 00000100:00100000:3.0:1789760794.192210:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778421965686 time 778462576745 state 0x0 ref 1 00000100:00100000:3.0:1789760794.192228:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778462837161 time 778462576745 state 0x1 ref 1 00000100:00100000:3.0:1789760794.192261:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778462837161 time 778512978275 state 0x0 ref 1 00000100:00100000:3.0:1789760794.192274:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fdea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495960704:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760794.192362:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28049408, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd809 00010000:00010000:3.0:1789760794.192371:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454830c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.192388:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=778s v=(0 0) 00010000:00010000:3.0:1789760794.192393:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.192399:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454830c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28049408->28053503] (req 28049408->28053503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.192418:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454830c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28049408->28053503] (req 28049408->28053503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.236839:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974165 is committed 00002000:00010000:1.0:1789760794.236941:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28049408 -> 28053504 00010000:00010000:1.0:1789760794.236956:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454830c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28049408->28053503] (req 28049408->28053503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.236968:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454830c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28049408->28053503] (req 28049408->28053503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.236977:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454830c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28049408->28053503] (req 28049408->28053503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.236984:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760794.236989:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454830c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28049408->28053503] (req 28049408->28053503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.236999:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454830c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28049408->28053503] (req 28049408->28053503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.237019:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454830c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28049408->28053503] (req 28049408->28053503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.237026:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454830c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28049408->28053503] (req 28049408->28053503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.237117:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fdea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495960704:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44841us (45069us total) trans 4294974165 rc 0/0 00000100:00100000:1.0:1789760794.237127:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6877 00000100:00100000:1.0:1789760794.237137:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778462837161 time 778512978275 state 0x2 ref 0 00000100:00100000:0.0:1789760794.241751:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760794.241824:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495960832 00000100:00100000:1.0:1789760794.241833:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495960832 00000100:00100000:1.0:1789760794.241888:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778462837161 time 778512978275 state 0x0 ref 1 00000100:00100000:1.0:1789760794.241903:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778513238691 time 778512978275 state 0x1 ref 1 00000100:00100000:1.0:1789760794.241931:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778513238691 time 778562648426 state 0x0 ref 1 00000100:00100000:1.0:1789760794.241946:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01377100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495960832:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760794.241998:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28053504, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd809 00010000:00010000:1.0:1789760794.242009:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548313 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.242023:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760794.242027:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.242033:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548313 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28053504->28057599] (req 28053504->28057599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.242047:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548313 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28053504->28057599] (req 28053504->28057599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.266483:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974166 is committed 00002000:00010000:1.0:1789760794.266593:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28053504 -> 28057600 00010000:00010000:1.0:1789760794.266608:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548313 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28053504->28057599] (req 28053504->28057599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.266621:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548313 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28053504->28057599] (req 28053504->28057599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.266630:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548313 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28053504->28057599] (req 28053504->28057599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.266638:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760794.266643:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548313 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28053504->28057599] (req 28053504->28057599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.266661:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548313 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28053504->28057599] (req 28053504->28057599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.266694:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548313 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28053504->28057599] (req 28053504->28057599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.266701:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548313 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28053504->28057599] (req 28053504->28057599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.266792:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01377100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495960832:12345-192.168.203.9@tcp:4:dd.0 Request processed in 24842us (25046us total) trans 4294974166 rc 0/0 00000100:00100000:1.0:1789760794.266802:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6878 00000100:00100000:1.0:1789760794.266811:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778513238691 time 778562648426 state 0x2 ref 0 00000100:00100000:2.0:1789760794.271359:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760794.273459:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495960960 00000100:00100000:1.0:1789760794.273472:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495960960 00000100:00100000:1.0:1789760794.273533:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778513238691 time 778562648426 state 0x0 ref 1 00000100:00100000:1.0:1789760794.273549:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778562908842 time 778562648426 state 0x1 ref 1 00000100:00100000:1.0:1789760794.273578:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778562908842 time 778594295597 state 0x0 ref 1 00000100:00100000:1.0:1789760794.273591:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fdf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495960960:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760794.273902:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28057600, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd809 00010000:00010000:1.0:1789760794.273912:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454831a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.273927:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760794.273932:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.273939:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454831a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28057600->28061695] (req 28057600->28061695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.273953:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454831a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28057600->28061695] (req 28057600->28061695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.310384:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974167 is committed 00002000:00010000:1.0:1789760794.310436:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28057600 -> 28061696 00010000:00010000:1.0:1789760794.310452:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454831a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28057600->28061695] (req 28057600->28061695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.310465:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454831a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28057600->28061695] (req 28057600->28061695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.310474:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454831a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28057600->28061695] (req 28057600->28061695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.310480:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760794.310485:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454831a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28057600->28061695] (req 28057600->28061695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.310493:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454831a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28057600->28061695] (req 28057600->28061695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.310512:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454831a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28057600->28061695] (req 28057600->28061695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.310525:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454831a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28057600->28061695] (req 28057600->28061695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.310678:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fdf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495960960:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37085us (39325us total) trans 4294974167 rc 0/0 00000100:00100000:1.0:1789760794.310690:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6879 00000100:00100000:1.0:1789760794.310700:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778562908842 time 778594295597 state 0x2 ref 0 00000100:00100000:2.0:1789760794.321534:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760794.321608:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495961088 00000100:00100000:1.0:1789760794.321618:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495961088 00000100:00100000:1.0:1789760794.321681:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778562908842 time 778594295597 state 0x0 ref 1 00000100:00100000:1.0:1789760794.321697:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778594556013 time 778594295597 state 0x1 ref 1 00000100:00100000:1.0:1789760794.321730:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778594556013 time 778642447204 state 0x0 ref 1 00000100:00100000:1.0:1789760794.321743:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fdd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495961088:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760794.321792:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28061696, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd809 00010000:00010000:1.0:1789760794.321801:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354548321 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.321817:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760794.321822:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.321828:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354548321 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28061696->28065791] (req 28061696->28065791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.321846:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354548321 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28061696->28065791] (req 28061696->28065791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.366117:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974168 is committed 00002000:00010000:1.0:1789760794.366204:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28061696 -> 28065792 00010000:00010000:1.0:1789760794.366221:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354548321 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28061696->28065791] (req 28061696->28065791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.366234:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354548321 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28061696->28065791] (req 28061696->28065791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.366244:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354548321 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28061696->28065791] (req 28061696->28065791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.366252:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760794.366257:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354548321 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28061696->28065791] (req 28061696->28065791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.366266:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354548321 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28061696->28065791] (req 28061696->28065791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.366305:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354548321 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28061696->28065791] (req 28061696->28065791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.366314:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354548321 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28061696->28065791] (req 28061696->28065791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.366454:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fdd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495961088:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44708us (44923us total) trans 4294974168 rc 0/0 00000100:00100000:1.0:1789760794.366526:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6880 00000100:00100000:1.0:1789760794.366537:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778594556013 time 778642447204 state 0x2 ref 0 00000100:00100000:1.0:1789760794.382112:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760794.385831:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495961216 00000100:00100000:3.0:1789760794.385847:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495961216 00000100:00100000:3.0:1789760794.385916:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778594556013 time 778642447204 state 0x0 ref 1 00000100:00100000:3.0:1789760794.385934:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778642707620 time 778642447204 state 0x1 ref 1 00000100:00100000:3.0:1789760794.386053:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778642707620 time 778706768990 state 0x0 ref 1 00000100:00100000:3.0:1789760794.386068:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495961216:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760794.386123:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28065792, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd809 00010000:00010000:3.0:1789760794.386133:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548328 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.386150:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=778s v=(0 0) 00010000:00010000:3.0:1789760794.386155:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.386161:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548328 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28065792->28069887] (req 28065792->28069887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.386174:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548328 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28065792->28069887] (req 28065792->28069887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.399426:0:3319:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9290404ad180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520318080:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760794.399496:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760794.400110:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520318080 00000100:00100000:3.0:1789760794.400119:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520318080 00000100:00100000:3.0:1789760794.400166:0:6711:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 220 00000100:00100000:3.0:1789760794.400175:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520318080:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760794.400207:0:6711:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760794.400257:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1876699520318080:12345-0@lo:13:osp-pre-1-0.0 Request processed in 82us (761us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760794.400264:0:6711:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 220 00000100:00100000:3.0:1789760794.400331:0:3319:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9290404ad180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3319:1876699520318080:0@lo:13:osp-pre-1-0.0 00000001:00080000:0.0:1789760794.430184:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974169 is committed 00002000:00010000:1.0:1789760794.430457:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28065792 -> 28069888 00010000:00010000:1.0:1789760794.430475:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548328 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28065792->28069887] (req 28065792->28069887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.430487:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548328 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28065792->28069887] (req 28065792->28069887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.430497:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548328 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28065792->28069887] (req 28065792->28069887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.430505:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760794.430510:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548328 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28065792->28069887] (req 28065792->28069887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.430519:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548328 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28065792->28069887] (req 28065792->28069887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.430540:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548328 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28065792->28069887] (req 28065792->28069887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.430554:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548328 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28065792->28069887] (req 28065792->28069887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.430827:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495961216:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44686us (48648us total) trans 4294974169 rc 0/0 00000100:00100000:1.0:1789760794.430838:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6881 00000100:00100000:1.0:1789760794.430848:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778642707620 time 778706768990 state 0x2 ref 0 00000100:00100000:3.0:1789760794.434757:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760794.435167:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495961344 00000100:00100000:1.0:1789760794.435177:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495961344 00000100:00100000:1.0:1789760794.435230:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778642707620 time 778706768990 state 0x0 ref 1 00000100:00100000:1.0:1789760794.435245:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778707029406 time 778706768990 state 0x1 ref 1 00000100:00100000:1.0:1789760794.435358:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778707029406 time 778756073734 state 0x0 ref 1 00000100:00100000:1.0:1789760794.435377:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495961344:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760794.435427:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28069888, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd809 00010000:00010000:1.0:1789760794.435436:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454832f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.435451:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760794.435456:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.435463:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454832f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28069888->28073983] (req 28069888->28073983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.435482:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454832f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28069888->28073983] (req 28069888->28073983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.462473:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760794.472508:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974170 is committed 00002000:00010000:0.0:1789760794.472602:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28069888 -> 28073984 00010000:00010000:0.0:1789760794.472617:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454832f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28069888->28073983] (req 28069888->28073983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.472702:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454832f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28069888->28073983] (req 28069888->28073983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.472718:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454832f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28069888->28073983] (req 28069888->28073983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.472725:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760794.472731:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454832f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28069888->28073983] (req 28069888->28073983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.472740:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454832f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28069888->28073983] (req 28069888->28073983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.472903:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454832f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28069888->28073983] (req 28069888->28073983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.472912:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x493674835454832f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28069888->28073983] (req 28069888->28073983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.473001:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495961344:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37627us (38249us total) trans 4294974170 rc 0/0 00000100:00100000:0.0:1789760794.473011:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6882 00000100:00100000:0.0:1789760794.473020:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778707029406 time 778756073734 state 0x2 ref 0 00000100:00100000:2.0:1789760794.479160:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760794.479264:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495961472 00000100:00100000:1.0:1789760794.479274:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495961472 00000100:00100000:1.0:1789760794.479358:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778707029406 time 778756073734 state 0x0 ref 1 00000100:00100000:1.0:1789760794.479375:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778756334150 time 778756073734 state 0x1 ref 1 00000100:00100000:1.0:1789760794.479397:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778756334150 time 778800113762 state 0x0 ref 1 00000100:00100000:1.0:1789760794.479408:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495961472:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760794.479458:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28073984, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd809 00010000:00010000:1.0:1789760794.479467:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354548336 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.479485:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760794.479489:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.479496:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354548336 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28073984->28078079] (req 28073984->28078079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.479510:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354548336 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28073984->28078079] (req 28073984->28078079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.508922:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974171 is committed 00002000:00010000:3.0:1789760794.509017:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28073984 -> 28078080 00010000:00010000:3.0:1789760794.509033:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354548336 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28073984->28078079] (req 28073984->28078079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.509045:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354548336 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28073984->28078079] (req 28073984->28078079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.509054:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354548336 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28073984->28078079] (req 28073984->28078079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.509060:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760794.509065:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354548336 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28073984->28078079] (req 28073984->28078079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.509076:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354548336 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28073984->28078079] (req 28073984->28078079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.509096:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354548336 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28073984->28078079] (req 28073984->28078079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.509103:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354548336 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28073984->28078079] (req 28073984->28078079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.509195:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495961472:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29784us (30042us total) trans 4294974171 rc 0/0 00000100:00100000:3.0:1789760794.509204:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6883 00000100:00100000:3.0:1789760794.509215:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778756334150 time 778800113762 state 0x2 ref 0 00000100:00100000:1.0:1789760794.514795:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760794.514950:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495961600 00000100:00100000:2.0:1789760794.514960:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495961600 00000100:00100000:2.0:1789760794.515007:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778756334150 time 778800113762 state 0x0 ref 1 00000100:00100000:2.0:1789760794.515022:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778800374178 time 778800113762 state 0x1 ref 1 00000100:00100000:2.0:1789760794.515091:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778800374178 time 778835806939 state 0x0 ref 1 00000100:00100000:2.0:1789760794.515102:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495961600:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760794.515147:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28078080, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd809 00010000:00010000:2.0:1789760794.515156:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454833d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.515181:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=778s v=(0 0) 00010000:00010000:2.0:1789760794.515186:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.515193:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454833d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28078080->28082175] (req 28078080->28082175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.515207:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454833d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28078080->28082175] (req 28078080->28082175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760794.555309:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28078080 -> 28082176 00010000:00010000:1.0:1789760794.555329:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454833d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28078080->28082175] (req 28078080->28082175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.555347:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454833d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28078080->28082175] (req 28078080->28082175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.555356:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454833d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28078080->28082175] (req 28078080->28082175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.555364:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760794.555369:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454833d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28078080->28082175] (req 28078080->28082175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.555378:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454833d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28078080->28082175] (req 28078080->28082175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.555397:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454833d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28078080->28082175] (req 28078080->28082175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.555405:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454833d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28078080->28082175] (req 28078080->28082175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.555489:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495961600:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40382us (40698us total) trans 4294974172 rc 0/0 00000100:00100000:1.0:1789760794.555500:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6884 00000100:00100000:1.0:1789760794.555510:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778800374178 time 778835806939 state 0x2 ref 0 00000001:00080000:3.0:1789760794.555599:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974172 is committed 00000100:00100000:1.0:1789760794.576166:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760794.576713:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495961728 00000100:00100000:0.0:1789760794.576724:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495961728 00000100:00100000:0.0:1789760794.576784:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778800374178 time 778835806939 state 0x0 ref 1 00000100:00100000:0.0:1789760794.576803:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778836067355 time 778835806939 state 0x1 ref 1 00000100:00100000:0.0:1789760794.576837:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778836067355 time 778897553890 state 0x0 ref 1 00000100:00100000:0.0:1789760794.576851:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495961728:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760794.576901:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28082176, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd809 00010000:00010000:0.0:1789760794.576909:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354548344 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.576925:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=778s v=(0 0) 00010000:00010000:0.0:1789760794.576929:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.576935:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354548344 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28082176->28086271] (req 28082176->28086271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.576948:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354548344 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28082176->28086271] (req 28082176->28086271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.616589:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974173 is committed 00002000:00010000:1.0:1789760794.616765:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28082176 -> 28086272 00010000:00010000:1.0:1789760794.616782:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354548344 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28082176->28086271] (req 28082176->28086271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.616795:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354548344 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28082176->28086271] (req 28082176->28086271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.616805:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354548344 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28082176->28086271] (req 28082176->28086271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.616812:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760794.616817:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354548344 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28082176->28086271] (req 28082176->28086271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.616827:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354548344 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28082176->28086271] (req 28082176->28086271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.616845:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354548344 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28082176->28086271] (req 28082176->28086271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.616854:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354548344 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28082176->28086271] (req 28082176->28086271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.617058:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495961728:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40205us (40895us total) trans 4294974173 rc 0/0 00000100:00100000:1.0:1789760794.617068:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6885 00000100:00100000:1.0:1789760794.617078:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778836067355 time 778897553890 state 0x2 ref 0 00000100:00100000:0.0:1789760794.619465:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760794.619539:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495961856 00000100:00100000:3.0:1789760794.619548:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495961856 00000100:00100000:3.0:1789760794.619597:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778836067355 time 778897553890 state 0x0 ref 1 00000100:00100000:3.0:1789760794.619612:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778897814306 time 778897553890 state 0x1 ref 1 00000100:00100000:3.0:1789760794.619693:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778897814306 time 778940409240 state 0x0 ref 1 00000100:00100000:3.0:1789760794.619706:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01375500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495961856:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760794.619762:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28086272, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd809 00010000:00010000:3.0:1789760794.619772:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454834b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.619792:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=778s v=(0 0) 00010000:00010000:3.0:1789760794.619796:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.619804:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454834b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28086272->28090367] (req 28086272->28090367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.619816:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454834b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28086272->28090367] (req 28086272->28090367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.667797:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974174 is committed 00002000:00010000:3.0:1789760794.667956:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28086272 -> 28090368 00002000:00010000:3.0:1789760794.667961:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760793 -> 1789760794 00002000:00010000:3.0:1789760794.667964:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760793 -> 1789760794 00010000:00010000:3.0:1789760794.667976:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454834b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28086272->28090367] (req 28086272->28090367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.667988:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454834b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28086272->28090367] (req 28086272->28090367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.667997:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454834b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28086272->28090367] (req 28086272->28090367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.668004:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760794.668009:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454834b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28086272->28090367] (req 28086272->28090367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.668018:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454834b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28086272->28090367] (req 28086272->28090367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.668037:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454834b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28086272->28090367] (req 28086272->28090367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.668046:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454834b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28086272->28090367] (req 28086272->28090367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.668262:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01375500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495961856:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48554us (48799us total) trans 4294974174 rc 0/0 00000100:00100000:3.0:1789760794.668273:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6886 00000100:00100000:3.0:1789760794.668321:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778897814306 time 778940409240 state 0x2 ref 0 00000100:00100000:0.0:1789760794.671461:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760794.671623:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495961984 00000100:00100000:1.0:1789760794.671634:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495961984 00000100:00100000:1.0:1789760794.671752:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778897814306 time 778940409240 state 0x0 ref 1 00000100:00100000:1.0:1789760794.671768:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778940669656 time 778940409240 state 0x1 ref 1 00000100:00100000:1.0:1789760794.671849:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778940669656 time 778992565055 state 0x0 ref 1 00000100:00100000:1.0:1789760794.671863:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01377480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495961984:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760794.671916:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28090368, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd809 00010000:00010000:1.0:1789760794.671925:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548352 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.671941:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760794.671945:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.671951:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548352 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28090368->28094463] (req 28090368->28094463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.671963:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548352 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28090368->28094463] (req 28090368->28094463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.719152:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974175 is committed 00002000:00010000:1.0:1789760794.720998:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28090368 -> 28094464 00010000:00010000:1.0:1789760794.721019:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548352 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28090368->28094463] (req 28090368->28094463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.721037:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548352 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28090368->28094463] (req 28090368->28094463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.721047:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548352 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28090368->28094463] (req 28090368->28094463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.721054:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760794.721060:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548352 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28090368->28094463] (req 28090368->28094463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.721069:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548352 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28090368->28094463] (req 28090368->28094463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.721091:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548352 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28090368->28094463] (req 28090368->28094463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.721100:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548352 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28090368->28094463] (req 28090368->28094463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.721224:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01377480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495961984:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49358us (49766us total) trans 4294974175 rc 0/0 00000100:00100000:1.0:1789760794.721238:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6887 00000100:00100000:1.0:1789760794.721249:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778940669656 time 778992565055 state 0x2 ref 0 00000100:00100000:2.0:1789760794.726524:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760794.726752:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495962112 00000100:00100000:1.0:1789760794.726762:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495962112 00000100:00100000:1.0:1789760794.726814:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778940669656 time 778992565055 state 0x0 ref 1 00000100:00100000:1.0:1789760794.726829:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778992825471 time 778992565055 state 0x1 ref 1 00000100:00100000:1.0:1789760794.727408:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778992825471 time 779048121808 state 0x0 ref 1 00000100:00100000:1.0:1789760794.727425:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495962112:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760794.727474:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28094464, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd809 00010000:00010000:1.0:1789760794.727483:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548359 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.727501:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760794.727506:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.727512:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548359 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28094464->28098559] (req 28094464->28098559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.727526:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548359 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28094464->28098559] (req 28094464->28098559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760794.789270:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28094464 -> 28098560 00010000:00010000:2.0:1789760794.789310:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548359 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28094464->28098559] (req 28094464->28098559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.789322:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548359 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28094464->28098559] (req 28094464->28098559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.789337:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548359 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28094464->28098559] (req 28094464->28098559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.789345:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760794.789349:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548359 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28094464->28098559] (req 28094464->28098559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.789357:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548359 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28094464->28098559] (req 28094464->28098559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.789377:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548359 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28094464->28098559] (req 28094464->28098559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.789384:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x4936748354548359 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28094464->28098559] (req 28094464->28098559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.789467:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495962112:12345-192.168.203.9@tcp:4:dd.0 Request processed in 62041us (62948us total) trans 4294974176 rc 0/0 00000100:00100000:2.0:1789760794.789477:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6888 00000100:00100000:2.0:1789760794.789488:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778992825471 time 779048121808 state 0x2 ref 0 00000001:00080000:2.0:1789760794.789531:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974176 is committed 00000100:00100000:0.0:1789760794.802065:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760794.802483:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495962240 00000100:00100000:3.0:1789760794.802494:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495962240 00000100:00100000:3.0:1789760794.802550:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 778992825471 time 779048121808 state 0x0 ref 1 00000100:00100000:3.0:1789760794.802568:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779048382224 time 779048121808 state 0x1 ref 1 00000100:00100000:3.0:1789760794.802658:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779048382224 time 779123374255 state 0x0 ref 1 00000100:00100000:3.0:1789760794.802672:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01376680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495962240:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760794.802720:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28098560, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd809 00010000:00010000:3.0:1789760794.802730:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548360 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.802748:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=779s v=(0 0) 00010000:00010000:3.0:1789760794.802755:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.802762:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548360 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28098560->28102655] (req 28098560->28102655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.802776:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548360 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28098560->28102655] (req 28098560->28102655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.827540:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974177 is committed 00002000:00010000:3.0:1789760794.827777:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28098560 -> 28102656 00010000:00010000:3.0:1789760794.827791:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548360 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28098560->28102655] (req 28098560->28102655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.827804:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548360 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28098560->28102655] (req 28098560->28102655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.827812:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548360 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28098560->28102655] (req 28098560->28102655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.827819:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760794.827823:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548360 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28098560->28102655] (req 28098560->28102655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.827830:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548360 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28098560->28102655] (req 28098560->28102655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.827849:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548360 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28098560->28102655] (req 28098560->28102655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.827856:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548360 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28098560->28102655] (req 28098560->28102655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.827984:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01376680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495962240:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25310us (25926us total) trans 4294974177 rc 0/0 00000100:00100000:3.0:1789760794.827994:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6889 00000100:00100000:3.0:1789760794.828004:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779048382224 time 779123374255 state 0x2 ref 0 00000100:00100000:0.0:1789760794.835017:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760794.835113:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495962368 00000100:00100000:1.0:1789760794.835124:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495962368 00000100:00100000:1.0:1789760794.835177:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779048382224 time 779123374255 state 0x0 ref 1 00000100:00100000:1.0:1789760794.835192:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779123634671 time 779123374255 state 0x1 ref 1 00000100:00100000:1.0:1789760794.835220:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779123634671 time 779155937618 state 0x0 ref 1 00000100:00100000:1.0:1789760794.835364:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01376d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495962368:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760794.835411:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28102656, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd809 00010000:00010000:1.0:1789760794.835421:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354548367 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.835437:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760794.835441:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.835448:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354548367 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28102656->28106751] (req 28102656->28106751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.835463:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354548367 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28102656->28106751] (req 28102656->28106751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.860940:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974178 is committed 00002000:00010000:1.0:1789760794.861053:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28102656 -> 28106752 00010000:00010000:1.0:1789760794.861069:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354548367 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28102656->28106751] (req 28102656->28106751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.861081:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354548367 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28102656->28106751] (req 28102656->28106751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.861097:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354548367 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28102656->28106751] (req 28102656->28106751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.861104:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760794.861109:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354548367 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28102656->28106751] (req 28102656->28106751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.861117:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354548367 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28102656->28106751] (req 28102656->28106751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.861137:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354548367 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28102656->28106751] (req 28102656->28106751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.861144:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354548367 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28102656->28106751] (req 28102656->28106751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.861234:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01376d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495962368:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25868us (26222us total) trans 4294974178 rc 0/0 00000100:00100000:1.0:1789760794.861245:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6890 00000100:00100000:1.0:1789760794.861254:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779123634671 time 779155937618 state 0x2 ref 0 00000100:00100000:2.0:1789760794.864473:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760794.864611:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495962496 00000100:00100000:0.0:1789760794.864621:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495962496 00000100:00100000:0.0:1789760794.864736:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779123634671 time 779155937618 state 0x0 ref 1 00000100:00100000:0.0:1789760794.864751:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779156198034 time 779155937618 state 0x1 ref 1 00000100:00100000:0.0:1789760794.864781:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779156198034 time 779185497403 state 0x0 ref 1 00000100:00100000:0.0:1789760794.864793:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495962496:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760794.864838:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28106752, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd809 00010000:00010000:0.0:1789760794.864847:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454836e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.864863:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=779s v=(0 0) 00010000:00010000:0.0:1789760794.864868:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.864875:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454836e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28106752->28110847] (req 28106752->28110847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.864889:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454836e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28106752->28110847] (req 28106752->28110847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760794.890091:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28106752 -> 28110848 00010000:00010000:1.0:1789760794.890110:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454836e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28106752->28110847] (req 28106752->28110847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.890127:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454836e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28106752->28110847] (req 28106752->28110847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.890135:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454836e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28106752->28110847] (req 28106752->28110847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.890142:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760794.890147:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454836e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28106752->28110847] (req 28106752->28110847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.890156:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454836e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28106752->28110847] (req 28106752->28110847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.890176:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454836e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28106752->28110847] (req 28106752->28110847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.890183:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454836e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28106752->28110847] (req 28106752->28110847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.890273:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495962496:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25477us (25805us total) trans 4294974179 rc 0/0 00000100:00100000:1.0:1789760794.890320:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6891 00000100:00100000:1.0:1789760794.890331:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779156198034 time 779185497403 state 0x2 ref 0 00000001:00080000:0.0:1789760794.890878:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974179 is committed 00000100:00100000:0.0:1789760794.896860:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760794.899176:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495962624 00000100:00100000:3.0:1789760794.899189:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495962624 00000100:00100000:2.0:1789760794.900697:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779156198034 time 779185497403 state 0x0 ref 1 00000100:00100000:2.0:1789760794.900723:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779185757819 time 779185497403 state 0x1 ref 1 00000100:00100000:2.0:1789760794.900808:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779185757819 time 779221523765 state 0x0 ref 1 00000100:00100000:2.0:1789760794.900824:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01374000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495962624:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760794.900875:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28110848, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd809 00010000:00010000:2.0:1789760794.900885:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548375 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.900901:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760794.900906:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.900913:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548375 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28110848->28114943] (req 28110848->28114943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.900928:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548375 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28110848->28114943] (req 28110848->28114943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.944594:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974180 is committed 00002000:00010000:3.0:1789760794.944675:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28110848 -> 28114944 00010000:00010000:3.0:1789760794.944691:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548375 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28110848->28114943] (req 28110848->28114943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.944861:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548375 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28110848->28114943] (req 28110848->28114943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.944873:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548375 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28110848->28114943] (req 28110848->28114943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.944892:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760794.944898:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548375 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28110848->28114943] (req 28110848->28114943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.944908:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548375 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28110848->28114943] (req 28110848->28114943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.944935:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548375 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28110848->28114943] (req 28110848->28114943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.944948:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecb43c34/0x4936748354548375 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28110848->28114943] (req 28110848->28114943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.945275:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01374000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495962624:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44214us (48183us total) trans 4294974180 rc 0/0 00000100:00100000:3.0:1789760794.945511:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6892 00000100:00100000:3.0:1789760794.945522:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779185757819 time 779221523765 state 0x2 ref 0 00000100:00100000:2.0:1789760794.953809:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760794.953982:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495962752 00000100:00100000:1.0:1789760794.953992:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495962752 00000100:00100000:1.0:1789760794.954048:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779185757819 time 779221523765 state 0x0 ref 1 00000100:00100000:1.0:1789760794.954064:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779221784181 time 779221523765 state 0x1 ref 1 00000100:00100000:1.0:1789760794.954094:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779221784181 time 779274811693 state 0x0 ref 1 00000100:00100000:1.0:1789760794.954107:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495962752:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760794.954158:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28114944, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd809 00010000:00010000:1.0:1789760794.954167:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454837c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.954182:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760794.954187:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.954194:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454837c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28114944->28119039] (req 28114944->28119039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.954208:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454837c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28114944->28119039] (req 28114944->28119039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.998124:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974181 is committed 00002000:00010000:1.0:1789760794.998186:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28114944 -> 28119040 00010000:00010000:1.0:1789760794.998201:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454837c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28114944->28119039] (req 28114944->28119039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.998212:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454837c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28114944->28119039] (req 28114944->28119039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.998225:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454837c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28114944->28119039] (req 28114944->28119039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.998233:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760794.998238:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454837c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28114944->28119039] (req 28114944->28119039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.998248:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454837c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28114944->28119039] (req 28114944->28119039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.998267:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454837c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28114944->28119039] (req 28114944->28119039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.998276:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x493674835454837c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28114944->28119039] (req 28114944->28119039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.998507:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fdf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495962752:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44268us (44572us total) trans 4294974181 rc 0/0 00000100:00100000:1.0:1789760794.998518:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6893 00000100:00100000:1.0:1789760794.998528:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779221784181 time 779274811693 state 0x2 ref 0 00000100:00100000:2.0:1789760795.004563:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760795.005252:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495962880 00000100:00100000:1.0:1789760795.005261:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495962880 00000100:00100000:1.0:1789760795.005403:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779221784181 time 779274811693 state 0x0 ref 1 00000100:00100000:1.0:1789760795.005416:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779275072109 time 779274811693 state 0x1 ref 1 00000100:00100000:1.0:1789760795.005437:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779275072109 time 779326154529 state 0x0 ref 1 00000100:00100000:1.0:1789760795.005448:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495962880:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760795.005493:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28119040, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd809 00010000:00010000:1.0:1789760795.005502:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354548383 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.005520:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760795.005523:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.005530:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354548383 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28119040->28123135] (req 28119040->28123135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.005543:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354548383 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28119040->28123135] (req 28119040->28123135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760795.006539:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789760795.033343:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28119040 -> 28123136 00010000:00010000:0.0:1789760795.033365:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354548383 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28119040->28123135] (req 28119040->28123135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.033389:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354548383 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28119040->28123135] (req 28119040->28123135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.033405:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354548383 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28119040->28123135] (req 28119040->28123135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.033412:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760795.033417:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354548383 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28119040->28123135] (req 28119040->28123135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.033427:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354548383 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28119040->28123135] (req 28119040->28123135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.033447:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354548383 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28119040->28123135] (req 28119040->28123135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.033455:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a0f9d06c/0x4936748354548383 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28119040->28123135] (req 28119040->28123135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.033550:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495962880:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28099us (28991us total) trans 4294974182 rc 0/0 00000100:00100000:0.0:1789760795.033560:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6894 00000100:00100000:0.0:1789760795.033571:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779275072109 time 779326154529 state 0x2 ref 0 00000001:00080000:2.0:1789760795.033747:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974182 is committed 00000100:00100000:1.0:1789760795.039191:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760795.039431:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495963008 00000100:00100000:3.0:1789760795.039441:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495963008 00000100:00100000:3.0:1789760795.039487:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779275072109 time 779326154529 state 0x0 ref 1 00000100:00100000:3.0:1789760795.039502:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779326414945 time 779326154529 state 0x1 ref 1 00000100:00100000:3.0:1789760795.039565:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779326414945 time 779360280913 state 0x0 ref 1 00000100:00100000:3.0:1789760795.039578:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495963008:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760795.039619:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28123136, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd809 00010000:00010000:3.0:1789760795.039637:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454838a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.039661:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=779s v=(0 0) 00010000:00010000:3.0:1789760795.039665:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.039672:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454838a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28123136->28127231] (req 28123136->28127231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.039691:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454838a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28123136->28127231] (req 28123136->28127231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.062490:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974183 is committed 00002000:00010000:3.0:1789760795.062590:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28123136 -> 28127232 00010000:00010000:3.0:1789760795.062605:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454838a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28123136->28127231] (req 28123136->28127231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.062678:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454838a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28123136->28127231] (req 28123136->28127231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.062690:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454838a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28123136->28127231] (req 28123136->28127231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.062698:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760795.062703:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454838a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28123136->28127231] (req 28123136->28127231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.062712:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454838a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28123136->28127231] (req 28123136->28127231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.062731:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454838a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28123136->28127231] (req 28123136->28127231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.062739:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454838a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28123136->28127231] (req 28123136->28127231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.062873:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495963008:12345-192.168.203.9@tcp:4:dd.0 Request processed in 23293us (23685us total) trans 4294974183 rc 0/0 00000100:00100000:3.0:1789760795.062883:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6895 00000100:00100000:3.0:1789760795.062894:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779326414945 time 779360280913 state 0x2 ref 0 00000100:00100000:1.0:1789760795.067753:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760795.068842:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495963136 00000100:00100000:2.0:1789760795.068853:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495963136 00000100:00100000:2.0:1789760795.068908:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779326414945 time 779360280913 state 0x0 ref 1 00000100:00100000:2.0:1789760795.068924:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779360541329 time 779360280913 state 0x1 ref 1 00000100:00100000:2.0:1789760795.068949:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779360541329 time 779389666476 state 0x0 ref 1 00000100:00100000:2.0:1789760795.068961:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495963136:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760795.069003:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28127232, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd809 00010000:00010000:2.0:1789760795.069012:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548391 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.069027:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760795.069032:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.069038:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548391 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28127232->28131327] (req 28127232->28131327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.069051:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548391 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28127232->28131327] (req 28127232->28131327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.120113:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974184 is committed 00002000:00010000:1.0:1789760795.120157:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28127232 -> 28131328 00010000:00010000:1.0:1789760795.120173:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548391 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28127232->28131327] (req 28127232->28131327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.120186:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548391 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28127232->28131327] (req 28127232->28131327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.120195:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548391 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28127232->28131327] (req 28127232->28131327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.120251:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760795.120257:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548391 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28127232->28131327] (req 28127232->28131327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.120268:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548391 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28127232->28131327] (req 28127232->28131327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.120409:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548391 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28127232->28131327] (req 28127232->28131327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.120419:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548391 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28127232->28131327] (req 28127232->28131327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.120539:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495963136:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51573us (52789us total) trans 4294974184 rc 0/0 00000100:00100000:0.0:1789760795.120550:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6896 00000100:00100000:0.0:1789760795.120559:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779360541329 time 779389666476 state 0x2 ref 0 00000100:00100000:1.0:1789760795.126495:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760795.126544:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760795.126615:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495963392 00000100:00100000:2.0:1789760795.126625:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495963392 00000100:00100000:1.0:1789760795.126695:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760795.126709:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495963520 00000100:00100000:2.0:1789760795.126714:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495963520 00000100:00100000:1.0:1789760795.126732:0:12710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 535 00000100:00100000:1.0:1789760795.126760:0:12710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:17022965-1e4f-46fb-915c-61730096b8e2+8:2358:x1876699495963392:12345-192.168.203.9@tcp:400:kworker.0 00000100:00080000:0.0:1789760795.126771:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760795.126841:0:12710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:17022965-1e4f-46fb-915c-61730096b8e2+8:2358:x1876699495963392:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 99us (355us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760795.127181:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 536 00000100:00100000:2.0:1789760795.127190:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2358:x1876699495963520:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:2.0:1789760795.127228:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d3ad1ae7-10c4-4c94-ab44-aee2b86ddea9+5:2358:x1876699495963520:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 40us (535us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760795.127236:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 536 00000100:00100000:3.0:1789760795.128106:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495963264 00000100:00100000:3.0:1789760795.128115:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495963264 00000100:00100000:3.0:1789760795.128163:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.9@tcp, seq: 422 00000100:00100000:3.0:1789760795.128172:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290122cd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2358:x1876699495963264:12345-192.168.203.9@tcp:400:kworker.0 00000100:00100000:3.0:1789760795.128215:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290122cd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5292164f-12cc-4fea-b062-6f9c12f1be43+13:2358:x1876699495963264:12345-192.168.203.9@tcp:400:kworker.0 Request processed in 44us (1669us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760795.128221:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 422 00000100:00100000:2.0:1789760795.128437:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760795.128480:0:12710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.9@tcp, seq: 535 00000100:00100000:0.0:1789760795.128580:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495963648 00000100:00100000:0.0:1789760795.128586:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495963648 00000100:00100000:0.0:1789760795.128627:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779360541329 time 779389666476 state 0x0 ref 1 00000100:00100000:0.0:1789760795.128652:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779389926892 time 779389666476 state 0x1 ref 1 00000100:00100000:0.0:1789760795.128675:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779389926892 time 779449392366 state 0x0 ref 1 00000100:00100000:0.0:1789760795.128686:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495963648:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760795.128730:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28131328, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd809 00010000:00010000:0.0:1789760795.128739:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548398 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.128755:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=779s v=(0 0) 00010000:00010000:0.0:1789760795.128760:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.128767:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548398 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28131328->28135423] (req 28131328->28135423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.128781:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548398 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28131328->28135423] (req 28131328->28135423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.169321:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974185 is committed 00002000:00010000:1.0:1789760795.169549:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28131328 -> 28135424 00010000:00010000:1.0:1789760795.169564:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548398 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28131328->28135423] (req 28131328->28135423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.169583:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548398 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28131328->28135423] (req 28131328->28135423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.169593:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548398 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28131328->28135423] (req 28131328->28135423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.169602:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760795.169607:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548398 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28131328->28135423] (req 28131328->28135423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.169735:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548398 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28131328->28135423] (req 28131328->28135423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.169758:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548398 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28131328->28135423] (req 28131328->28135423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.169767:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548398 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28131328->28135423] (req 28131328->28135423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.169897:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495963648:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41206us (41455us total) trans 4294974185 rc 0/0 00000100:00100000:1.0:1789760795.169909:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6897 00000100:00100000:1.0:1789760795.169919:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779389926892 time 779449392366 state 0x2 ref 0 00000100:00100000:1.0:1789760795.174125:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760795.174270:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495963776 00000100:00100000:3.0:1789760795.174381:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495963776 00000100:00100000:3.0:1789760795.174435:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779389926892 time 779449392366 state 0x0 ref 1 00000100:00100000:3.0:1789760795.174451:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779449652782 time 779449392366 state 0x1 ref 1 00000100:00100000:3.0:1789760795.174559:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779449652782 time 779495274767 state 0x0 ref 1 00000100:00100000:3.0:1789760795.174573:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495963776:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760795.174699:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28135424, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd809 00010000:00010000:3.0:1789760795.174776:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454839f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.174794:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=779s v=(0 0) 00010000:00010000:3.0:1789760795.174799:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.174805:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454839f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28135424->28139519] (req 28135424->28139519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.174820:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454839f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28135424->28139519] (req 28135424->28139519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760795.202446:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28135424 -> 28139520 00010000:00010000:3.0:1789760795.202466:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454839f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28135424->28139519] (req 28135424->28139519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.202478:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454839f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28135424->28139519] (req 28135424->28139519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.202487:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454839f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28135424->28139519] (req 28135424->28139519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.202494:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760795.202499:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454839f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28135424->28139519] (req 28135424->28139519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.202508:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454839f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28135424->28139519] (req 28135424->28139519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.202527:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454839f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28135424->28139519] (req 28135424->28139519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.202536:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454839f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28135424->28139519] (req 28135424->28139519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.202731:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974186 is committed 00000100:00100000:3.0:1789760795.203110:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495963776:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28041us (28498us total) trans 4294974186 rc 0/0 00000100:00100000:3.0:1789760795.203124:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6898 00000100:00100000:3.0:1789760795.203134:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779449652782 time 779495274767 state 0x2 ref 0 00000100:00100000:0.0:1789760795.206075:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760795.206172:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495963904 00000100:00100000:2.0:1789760795.206182:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495963904 00000100:00100000:2.0:1789760795.206237:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779449652782 time 779495274767 state 0x0 ref 1 00000100:00100000:2.0:1789760795.206252:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779495535183 time 779495274767 state 0x1 ref 1 00000100:00100000:2.0:1789760795.206317:0:13059:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779495535183 time 779527033478 state 0x0 ref 1 00000100:00100000:2.0:1789760795.206329:0:13059:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495963904:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760795.206374:0:13059:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28139520, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd809 00010000:00010000:2.0:1789760795.206383:0:13059:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545483a6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.206400:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760795.206404:0:13059:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.206410:0:13059:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545483a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28139520->28143615] (req 28139520->28143615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.206424:0:13059:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545483a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28139520->28143615] (req 28139520->28143615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760795.239483:0:13059:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28139520 -> 28143616 00010000:00010000:3.0:1789760795.239502:0:13059:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545483a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28139520->28143615] (req 28139520->28143615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.239515:0:13059:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545483a6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28139520->28143615] (req 28139520->28143615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.239524:0:13059:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545483a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28139520->28143615] (req 28139520->28143615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.239532:0:13059:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760795.239537:0:13059:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545483a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28139520->28143615] (req 28139520->28143615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.239546:0:13059:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545483a6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28139520->28143615] (req 28139520->28143615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.239571:0:13059:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545483a6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28139520->28143615] (req 28139520->28143615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.239579:0:13059:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040be4b33/0x49367483545483a6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28139520->28143615] (req 28139520->28143615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13059 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.239685:0:13059:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01374380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495963904:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33354us (33615us total) trans 4294974187 rc 0/0 00000100:00100000:3.0:1789760795.239696:0:13059:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6899 00000100:00100000:3.0:1789760795.239713:0:13059:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779495535183 time 779527033478 state 0x2 ref 0 00000001:00080000:1.0:1789760795.240310:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974187 is committed 00000100:00100000:0.0:1789760795.242864:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760795.242947:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495964032 00000100:00100000:1.0:1789760795.242956:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495964032 00000100:00100000:1.0:1789760795.243014:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779495535183 time 779527033478 state 0x0 ref 1 00000100:00100000:1.0:1789760795.243028:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779527293894 time 779527033478 state 0x1 ref 1 00000100:00100000:1.0:1789760795.243052:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779527293894 time 779563769927 state 0x0 ref 1 00000100:00100000:1.0:1789760795.243063:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01374700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495964032:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760795.243107:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28143616, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd809 00010000:00010000:1.0:1789760795.243115:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545483ad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.243129:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760795.243134:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.243140:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545483ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28143616->28147711] (req 28143616->28147711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.243154:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545483ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28143616->28147711] (req 28143616->28147711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.297072:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974188 is committed 00002000:00010000:1.0:1789760795.297271:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28143616 -> 28147712 00010000:00010000:1.0:1789760795.297306:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545483ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28143616->28147711] (req 28143616->28147711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.297320:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545483ad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28143616->28147711] (req 28143616->28147711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.297330:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545483ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28143616->28147711] (req 28143616->28147711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.297338:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760795.297342:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545483ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28143616->28147711] (req 28143616->28147711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.297351:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545483ad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28143616->28147711] (req 28143616->28147711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.297371:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545483ad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28143616->28147711] (req 28143616->28147711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.297379:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545483ad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28143616->28147711] (req 28143616->28147711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.297466:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01374700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495964032:12345-192.168.203.9@tcp:4:dd.0 Request processed in 54399us (54605us total) trans 4294974188 rc 0/0 00000100:00100000:1.0:1789760795.297476:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6900 00000100:00100000:1.0:1789760795.297490:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779527293894 time 779563769927 state 0x2 ref 0 00000100:00100000:0.0:1789760795.303034:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760795.303190:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495964160 00000100:00100000:3.0:1789760795.303203:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495964160 00000100:00100000:3.0:1789760795.303259:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779527293894 time 779563769927 state 0x0 ref 1 00000100:00100000:3.0:1789760795.303274:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779564030343 time 779563769927 state 0x1 ref 1 00000100:00100000:3.0:1789760795.303348:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779564030343 time 779624064973 state 0x0 ref 1 00000100:00100000:3.0:1789760795.303360:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01375c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495964160:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760795.303412:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28147712, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd809 00010000:00010000:3.0:1789760795.303421:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545483b4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.303438:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=779s v=(0 0) 00010000:00010000:3.0:1789760795.303443:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.303449:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545483b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28147712->28151807] (req 28147712->28151807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.303465:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545483b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28147712->28151807] (req 28147712->28151807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.336825:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974189 is committed 00002000:00010000:1.0:1789760795.337080:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28147712 -> 28151808 00010000:00010000:1.0:1789760795.337095:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545483b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28147712->28151807] (req 28147712->28151807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.337115:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545483b4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28147712->28151807] (req 28147712->28151807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.337124:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545483b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28147712->28151807] (req 28147712->28151807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.337132:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760795.337136:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545483b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28147712->28151807] (req 28147712->28151807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.337145:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545483b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28147712->28151807] (req 28147712->28151807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.337226:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545483b4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28147712->28151807] (req 28147712->28151807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.337235:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545483b4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28147712->28151807] (req 28147712->28151807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.337414:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01375c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495964160:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34051us (34388us total) trans 4294974189 rc 0/0 00000100:00100000:1.0:1789760795.337693:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6901 00000100:00100000:1.0:1789760795.337714:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779564030343 time 779624064973 state 0x2 ref 0 00000100:00100000:0.0:1789760795.341870:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760795.341968:0:13059:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495964288 00000100:00100000:3.0:1789760795.341978:0:13059:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495964288 00000100:00100000:3.0:1789760795.342030:0:13059:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779564030343 time 779624064973 state 0x0 ref 1 00000100:00100000:3.0:1789760795.342044:0:13059:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779624325389 time 779624064973 state 0x1 ref 1 00000100:00100000:0.0:1789760795.342132:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779624325389 time 779662845934 state 0x0 ref 1 00000100:00100000:0.0:1789760795.342146:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01377800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495964288:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760795.342205:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28151808, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd809 00010000:00010000:0.0:1789760795.342214:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545483bb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.342232:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=779s v=(0 0) 00010000:00010000:0.0:1789760795.342236:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.342241:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545483bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28151808->28155903] (req 28151808->28155903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.342254:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545483bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28151808->28155903] (req 28151808->28155903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.388115:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974190 is committed 00002000:00010000:3.0:1789760795.388225:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28151808 -> 28155904 00010000:00010000:3.0:1789760795.388240:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545483bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28151808->28155903] (req 28151808->28155903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.388250:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545483bb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28151808->28155903] (req 28151808->28155903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.388256:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545483bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28151808->28155903] (req 28151808->28155903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.388260:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760795.388264:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545483bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28151808->28155903] (req 28151808->28155903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.388271:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545483bb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28151808->28155903] (req 28151808->28155903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.388344:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545483bb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28151808->28155903] (req 28151808->28155903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.388353:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db55e3c7/0x49367483545483bb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28151808->28155903] (req 28151808->28155903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.388459:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01377800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495964288:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46311us (46591us total) trans 4294974190 rc 0/0 00000100:00100000:3.0:1789760795.388470:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6902 00000100:00100000:3.0:1789760795.388480:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779624325389 time 779662845934 state 0x2 ref 0 00000100:00100000:2.0:1789760795.394845:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760795.395784:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495964416 00000100:00100000:1.0:1789760795.395796:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495964416 00000100:00100000:1.0:1789760795.395855:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779624325389 time 779662845934 state 0x0 ref 1 00000100:00100000:1.0:1789760795.395870:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779663106350 time 779662845934 state 0x1 ref 1 00000100:00100000:1.0:1789760795.398273:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779663106350 time 779718977365 state 0x0 ref 1 00000100:00100000:1.0:1789760795.398387:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495964416:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760795.398448:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28155904, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd809 00010000:00010000:1.0:1789760795.398457:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545483c2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.398475:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760795.398480:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.398486:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545483c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28155904->28159999] (req 28155904->28159999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.398501:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545483c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28155904->28159999] (req 28155904->28159999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.451370:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974191 is committed 00002000:00010000:3.0:1789760795.451491:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28155904 -> 28160000 00010000:00010000:3.0:1789760795.451507:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545483c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28155904->28159999] (req 28155904->28159999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.451521:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545483c2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28155904->28159999] (req 28155904->28159999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.451531:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545483c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28155904->28159999] (req 28155904->28159999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.451540:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760795.451545:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545483c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28155904->28159999] (req 28155904->28159999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.451558:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545483c2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28155904->28159999] (req 28155904->28159999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.451578:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545483c2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28155904->28159999] (req 28155904->28159999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.451585:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000068ad5ed1/0x49367483545483c2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28155904->28159999] (req 28155904->28159999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.451699:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495964416:12345-192.168.203.9@tcp:4:dd.0 Request processed in 53322us (56859us total) trans 4294974191 rc 0/0 00000100:00100000:3.0:1789760795.451709:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6903 00000100:00100000:3.0:1789760795.451719:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779663106350 time 779718977365 state 0x2 ref 0 00000100:00100000:0.0:1789760795.461394:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760795.461774:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495964544 00000100:00100000:1.0:1789760795.461784:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495964544 00000100:00100000:1.0:1789760795.461841:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779663106350 time 779718977365 state 0x0 ref 1 00000100:00100000:1.0:1789760795.461858:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779719237781 time 779718977365 state 0x1 ref 1 00000100:00100000:1.0:1789760795.461892:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779719237781 time 779782608851 state 0x0 ref 1 00000100:00100000:1.0:1789760795.461909:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01374e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495964544:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760795.461960:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28160000, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd809 00010000:00010000:1.0:1789760795.461969:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545483c9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.461985:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760795.461989:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.461996:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545483c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28160000->28164095] (req 28160000->28164095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.462009:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545483c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28160000->28164095] (req 28160000->28164095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.486835:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760795.508355:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974192 is committed 00002000:00010000:0.0:1789760795.508456:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28160000 -> 28164096 00010000:00010000:0.0:1789760795.508536:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545483c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28160000->28164095] (req 28160000->28164095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.508548:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545483c9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28160000->28164095] (req 28160000->28164095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.508556:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545483c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28160000->28164095] (req 28160000->28164095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.508564:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760795.508569:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545483c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28160000->28164095] (req 28160000->28164095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.508579:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545483c9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28160000->28164095] (req 28160000->28164095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.508599:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545483c9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28160000->28164095] (req 28160000->28164095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.508606:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x49367483545483c9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28160000->28164095] (req 28160000->28164095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.508864:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01374e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495964544:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46952us (47474us total) trans 4294974192 rc 0/0 00000100:00100000:0.0:1789760795.508874:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6904 00000100:00100000:0.0:1789760795.508885:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779719237781 time 779782608851 state 0x2 ref 0 00000100:00100000:0.0:1789760795.514733:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760795.514906:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495964672 00000100:00100000:3.0:1789760795.514917:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495964672 00000100:00100000:3.0:1789760795.514984:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779719237781 time 779782608851 state 0x0 ref 1 00000100:00100000:3.0:1789760795.515000:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779782869267 time 779782608851 state 0x1 ref 1 00000100:00100000:3.0:1789760795.515027:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779782869267 time 779835743874 state 0x0 ref 1 00000100:00100000:3.0:1789760795.515038:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495964672:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760795.515088:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28164096, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd809 00010000:00010000:3.0:1789760795.515097:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483d0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.515114:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=779s v=(0 0) 00010000:00010000:3.0:1789760795.515118:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.515125:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28164096->28168191] (req 28164096->28168191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.515148:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28164096->28168191] (req 28164096->28168191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760795.552628:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28164096 -> 28168192 00010000:00010000:3.0:1789760795.552656:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28164096->28168191] (req 28164096->28168191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.552670:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483d0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28164096->28168191] (req 28164096->28168191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.552680:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28164096->28168191] (req 28164096->28168191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.552689:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760795.552694:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28164096->28168191] (req 28164096->28168191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.552767:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483d0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28164096->28168191] (req 28164096->28168191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.552789:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483d0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28164096->28168191] (req 28164096->28168191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.552797:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483d0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28164096->28168191] (req 28164096->28168191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.552888:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495964672:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37846us (38162us total) trans 4294974193 rc 0/0 00000100:00100000:3.0:1789760795.552898:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6905 00000100:00100000:3.0:1789760795.552909:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779782869267 time 779835743874 state 0x2 ref 0 00000001:00080000:3.0:1789760795.552947:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974193 is committed 00000100:00100000:2.0:1789760795.558127:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760795.558366:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495964800 00000100:00100000:3.0:1789760795.558375:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495964800 00000100:00100000:2.0:1789760795.558567:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779782869267 time 779835743874 state 0x0 ref 1 00000100:00100000:2.0:1789760795.558582:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779836004290 time 779835743874 state 0x1 ref 1 00000100:00100000:2.0:1789760795.558658:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779836004290 time 779879374655 state 0x0 ref 1 00000100:00100000:2.0:1789760795.558671:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fdc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495964800:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760795.558725:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28168192, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd809 00010000:00010000:2.0:1789760795.558735:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545483d7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.558752:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760795.558757:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.558764:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545483d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28168192->28172287] (req 28168192->28172287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.558779:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545483d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28168192->28172287] (req 28168192->28172287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.621220:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974194 is committed 00002000:00010000:3.0:1789760795.621334:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28168192 -> 28172288 00010000:00010000:3.0:1789760795.621349:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545483d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28168192->28172287] (req 28168192->28172287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.621362:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545483d7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28168192->28172287] (req 28168192->28172287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.621372:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545483d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28168192->28172287] (req 28168192->28172287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.621380:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760795.621385:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545483d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28168192->28172287] (req 28168192->28172287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.621405:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545483d7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28168192->28172287] (req 28168192->28172287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.621425:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545483d7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28168192->28172287] (req 28168192->28172287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.621434:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x49367483545483d7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28168192->28172287] (req 28168192->28172287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.621555:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fdc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495964800:12345-192.168.203.9@tcp:4:dd.0 Request processed in 62880us (63435us total) trans 4294974194 rc 0/0 00000100:00100000:3.0:1789760795.621565:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6906 00000100:00100000:3.0:1789760795.621576:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779836004290 time 779879374655 state 0x2 ref 0 00000100:00100000:0.0:1789760795.626902:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760795.627046:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495964928 00000100:00100000:3.0:1789760795.627055:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495964928 00000100:00100000:3.0:1789760795.627117:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779836004290 time 779879374655 state 0x0 ref 1 00000100:00100000:3.0:1789760795.627131:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779879635071 time 779879374655 state 0x1 ref 1 00000100:00100000:3.0:1789760795.627508:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779879635071 time 779948222822 state 0x0 ref 1 00000100:00100000:3.0:1789760795.627522:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495964928:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760795.627566:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28172288, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd809 00010000:00010000:3.0:1789760795.627575:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483de lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.627591:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=779s v=(0 0) 00010000:00010000:3.0:1789760795.627595:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.627602:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28172288->28176383] (req 28172288->28176383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.627626:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28172288->28176383] (req 28172288->28176383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.682369:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974195 is committed 00002000:00010000:0.0:1789760795.682502:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28172288 -> 28176384 00002000:00010000:0.0:1789760795.682508:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760794 -> 1789760795 00002000:00010000:0.0:1789760795.682511:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760794 -> 1789760795 00010000:00010000:0.0:1789760795.682523:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28172288->28176383] (req 28172288->28176383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.682537:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483de lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28172288->28176383] (req 28172288->28176383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.682546:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28172288->28176383] (req 28172288->28176383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.682553:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760795.682558:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28172288->28176383] (req 28172288->28176383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.682568:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28172288->28176383] (req 28172288->28176383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.682592:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483de lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28172288->28176383] (req 28172288->28176383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.682601:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483de lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28172288->28176383] (req 28172288->28176383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.682801:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495964928:12345-192.168.203.9@tcp:4:dd.0 Request processed in 55277us (55903us total) trans 4294974195 rc 0/0 00000100:00100000:0.0:1789760795.682811:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6907 00000100:00100000:0.0:1789760795.682821:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779879635071 time 779948222822 state 0x2 ref 0 00000100:00100000:0.0:1789760795.686168:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760795.686320:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495965056 00000100:00100000:3.0:1789760795.686329:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495965056 00000100:00100000:3.0:1789760795.686380:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779879635071 time 779948222822 state 0x0 ref 1 00000100:00100000:3.0:1789760795.686395:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779948483238 time 779948222822 state 0x1 ref 1 00000100:00100000:3.0:1789760795.686428:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779948483238 time 780007144977 state 0x0 ref 1 00000100:00100000:3.0:1789760795.686441:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495965056:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760795.686489:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28176384, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd809 00010000:00010000:3.0:1789760795.686498:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545483e5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.686515:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760795.686520:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.686526:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545483e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28176384->28180479] (req 28176384->28180479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.686547:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545483e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28176384->28180479] (req 28176384->28180479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760795.721347:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28176384 -> 28180480 00010000:00010000:1.0:1789760795.721367:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545483e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28176384->28180479] (req 28176384->28180479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.721380:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545483e5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28176384->28180479] (req 28176384->28180479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.721390:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545483e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28176384->28180479] (req 28176384->28180479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.721397:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760795.721402:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545483e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28176384->28180479] (req 28176384->28180479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.721411:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545483e5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28176384->28180479] (req 28176384->28180479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.721431:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545483e5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28176384->28180479] (req 28176384->28180479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.721439:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545483e5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28176384->28180479] (req 28176384->28180479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.721539:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495965056:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35095us (35374us total) trans 4294974196 rc 0/0 00000100:00100000:1.0:1789760795.721549:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6908 00000100:00100000:1.0:1789760795.721559:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779948483238 time 780007144977 state 0x2 ref 0 00000001:00080000:2.0:1789760795.721793:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974196 is committed 00000100:00100000:2.0:1789760795.726685:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760795.726763:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495965184 00000100:00100000:3.0:1789760795.726773:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495965184 00000100:00100000:3.0:1789760795.726825:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 779948483238 time 780007144977 state 0x0 ref 1 00000100:00100000:3.0:1789760795.726840:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780007405393 time 780007144977 state 0x1 ref 1 00000100:00100000:3.0:1789760795.726868:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780007405393 time 780047585314 state 0x0 ref 1 00000100:00100000:3.0:1789760795.726880:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495965184:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760795.726922:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28180480, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd809 00010000:00010000:3.0:1789760795.726931:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483ec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.726946:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760795.726950:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.726957:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28180480->28184575] (req 28180480->28184575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.726981:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28180480->28184575] (req 28180480->28184575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.758992:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974197 is committed 00002000:00010000:2.0:1789760795.759131:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28180480 -> 28184576 00010000:00010000:2.0:1789760795.759146:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28180480->28184575] (req 28180480->28184575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.759158:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483ec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28180480->28184575] (req 28180480->28184575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.759167:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28180480->28184575] (req 28180480->28184575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.759173:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760795.759178:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28180480->28184575] (req 28180480->28184575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.759187:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483ec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28180480->28184575] (req 28180480->28184575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.759205:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483ec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28180480->28184575] (req 28180480->28184575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.759213:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x49367483545483ec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28180480->28184575] (req 28180480->28184575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.759333:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495965184:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32450us (32653us total) trans 4294974197 rc 0/0 00000100:00100000:2.0:1789760795.759344:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6909 00000100:00100000:2.0:1789760795.759354:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780007405393 time 780047585314 state 0x2 ref 0 00000100:00100000:0.0:1789760795.762083:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760795.762171:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495965312 00000100:00100000:3.0:1789760795.762182:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495965312 00000100:00100000:3.0:1789760795.762233:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780007405393 time 780047585314 state 0x0 ref 1 00000100:00100000:3.0:1789760795.762247:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780047845730 time 780047585314 state 0x1 ref 1 00000100:00100000:3.0:1789760795.762315:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780047845730 time 780082996707 state 0x0 ref 1 00000100:00100000:3.0:1789760795.762327:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495965312:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760795.762374:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28184576, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd809 00010000:00010000:3.0:1789760795.762383:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545483f3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.762399:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760795.762403:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.762409:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545483f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28184576->28188671] (req 28184576->28188671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.762421:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545483f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28184576->28188671] (req 28184576->28188671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760795.789222:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28184576 -> 28188672 00010000:00010000:0.0:1789760795.789241:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545483f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28184576->28188671] (req 28184576->28188671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.789254:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545483f3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28184576->28188671] (req 28184576->28188671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.789263:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545483f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28184576->28188671] (req 28184576->28188671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.789271:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760795.789306:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545483f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28184576->28188671] (req 28184576->28188671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.789317:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545483f3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28184576->28188671] (req 28184576->28188671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.789337:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545483f3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28184576->28188671] (req 28184576->28188671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.789345:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x49367483545483f3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28184576->28188671] (req 28184576->28188671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.789427:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495965312:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27097us (27348us total) trans 4294974198 rc 0/0 00000100:00100000:0.0:1789760795.789436:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6910 00000100:00100000:0.0:1789760795.789447:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780047845730 time 780082996707 state 0x2 ref 0 00000001:00080000:0.0:1789760795.790383:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974198 is committed 00000100:00100000:0.0:1789760795.793129:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760795.793210:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495965440 00000100:00100000:1.0:1789760795.793219:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495965440 00000100:00100000:1.0:1789760795.793268:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780047845730 time 780082996707 state 0x0 ref 1 00000100:00100000:1.0:1789760795.793307:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780083257123 time 780082996707 state 0x1 ref 1 00000100:00100000:1.0:1789760795.793335:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780083257123 time 780114052530 state 0x0 ref 1 00000100:00100000:1.0:1789760795.793346:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495965440:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760795.793388:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28188672, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd809 00010000:00010000:1.0:1789760795.793397:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545483fa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.793411:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760795.793416:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.793422:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545483fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28188672->28192767] (req 28188672->28192767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.793435:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545483fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28188672->28192767] (req 28188672->28192767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.821945:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974199 is committed 00002000:00010000:0.0:1789760795.822239:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28188672 -> 28192768 00010000:00010000:0.0:1789760795.822254:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545483fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28188672->28192767] (req 28188672->28192767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.822267:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545483fa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28188672->28192767] (req 28188672->28192767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.822277:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545483fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28188672->28192767] (req 28188672->28192767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.822346:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760795.822350:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545483fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28188672->28192767] (req 28188672->28192767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.822359:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545483fa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28188672->28192767] (req 28188672->28192767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.822377:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545483fa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28188672->28192767] (req 28188672->28192767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.822385:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b828ece7/0x49367483545483fa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28188672->28192767] (req 28188672->28192767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.823090:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f033a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495965440:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29739us (29963us total) trans 4294974199 rc 0/0 00000100:00100000:0.0:1789760795.823103:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6911 00000100:00100000:0.0:1789760795.823113:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780083257123 time 780114052530 state 0x2 ref 0 00000100:00100000:2.0:1789760795.830257:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760795.830455:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495965568 00000100:00100000:3.0:1789760795.830535:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495965568 00000100:00100000:3.0:1789760795.830732:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780083257123 time 780114052530 state 0x0 ref 1 00000100:00100000:3.0:1789760795.830747:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780114312946 time 780114052530 state 0x1 ref 1 00000100:00100000:3.0:1789760795.830812:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780114312946 time 780151528697 state 0x0 ref 1 00000100:00100000:3.0:1789760795.830829:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495965568:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760795.830876:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28192768, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd809 00010000:00010000:3.0:1789760795.830885:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548401 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.830900:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760795.830905:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.830910:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548401 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28192768->28196863] (req 28192768->28196863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.830923:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548401 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28192768->28196863] (req 28192768->28196863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.872308:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974200 is committed 00002000:00010000:2.0:1789760795.872520:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28192768 -> 28196864 00010000:00010000:2.0:1789760795.872533:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548401 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28192768->28196863] (req 28192768->28196863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.872545:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548401 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28192768->28196863] (req 28192768->28196863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.872553:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548401 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28192768->28196863] (req 28192768->28196863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.872565:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760795.872570:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548401 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28192768->28196863] (req 28192768->28196863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.872578:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548401 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28192768->28196863] (req 28192768->28196863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.872597:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548401 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28192768->28196863] (req 28192768->28196863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.872604:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548401 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28192768->28196863] (req 28192768->28196863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.872802:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495965568:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41971us (42550us total) trans 4294974200 rc 0/0 00000100:00100000:2.0:1789760795.872811:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6912 00000100:00100000:2.0:1789760795.872820:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780114312946 time 780151528697 state 0x2 ref 0 00000100:00100000:1.0:1789760795.876842:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760795.876923:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495965696 00000100:00100000:0.0:1789760795.876933:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495965696 00000100:00100000:0.0:1789760795.876996:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780114312946 time 780151528697 state 0x0 ref 1 00000100:00100000:0.0:1789760795.877011:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780151789113 time 780151528697 state 0x1 ref 1 00000100:00100000:0.0:1789760795.877038:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780151789113 time 780197755826 state 0x0 ref 1 00000100:00100000:0.0:1789760795.877050:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495965696:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760795.877094:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28196864, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd809 00010000:00010000:0.0:1789760795.877103:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354548408 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.877119:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=780s v=(0 0) 00010000:00010000:0.0:1789760795.877123:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.877129:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354548408 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28196864->28200959] (req 28196864->28200959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.877142:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354548408 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28196864->28200959] (req 28196864->28200959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.912232:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974201 is committed 00002000:00010000:1.0:1789760795.912566:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28196864 -> 28200960 00010000:00010000:1.0:1789760795.912583:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354548408 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28196864->28200959] (req 28196864->28200959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.912597:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354548408 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28196864->28200959] (req 28196864->28200959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.912606:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354548408 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28196864->28200959] (req 28196864->28200959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.912682:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760795.912688:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354548408 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28196864->28200959] (req 28196864->28200959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.912698:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354548408 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28196864->28200959] (req 28196864->28200959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.912718:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354548408 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28196864->28200959] (req 28196864->28200959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.912726:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x4936748354548408 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28196864->28200959] (req 28196864->28200959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.912872:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495965696:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35817us (36032us total) trans 4294974201 rc 0/0 00000100:00100000:1.0:1789760795.912882:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6913 00000100:00100000:1.0:1789760795.912892:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780151789113 time 780197755826 state 0x2 ref 0 00000100:00100000:0.0:1789760795.917903:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760795.918066:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495965824 00000100:00100000:1.0:1789760795.918079:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495965824 00000100:00100000:1.0:1789760795.918145:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780151789113 time 780197755826 state 0x0 ref 1 00000100:00100000:1.0:1789760795.918162:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780198016242 time 780197755826 state 0x1 ref 1 00000100:00100000:1.0:1789760795.918241:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780198016242 time 780238957277 state 0x0 ref 1 00000100:00100000:1.0:1789760795.918255:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495965824:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760795.918338:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28200960, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd809 00010000:00010000:1.0:1789760795.918346:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454840f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.918365:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760795.918370:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.918377:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454840f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28200960->28205055] (req 28200960->28205055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.918393:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454840f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28200960->28205055] (req 28200960->28205055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760795.967074:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28200960 -> 28205056 00010000:00010000:0.0:1789760795.967098:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454840f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28200960->28205055] (req 28200960->28205055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.967113:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454840f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28200960->28205055] (req 28200960->28205055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.967122:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454840f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28200960->28205055] (req 28200960->28205055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.967129:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760795.967134:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454840f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28200960->28205055] (req 28200960->28205055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.967144:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454840f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28200960->28205055] (req 28200960->28205055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.967166:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454840f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28200960->28205055] (req 28200960->28205055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.967178:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e804abc/0x493674835454840f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28200960->28205055] (req 28200960->28205055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.967361:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495965824:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49103us (49465us total) trans 4294974202 rc 0/0 00000100:00100000:0.0:1789760795.967373:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6914 00000100:00100000:0.0:1789760795.967383:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780198016242 time 780238957277 state 0x2 ref 0 00000001:00080000:3.0:1789760795.967389:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974202 is committed 00000100:00100000:2.0:1789760795.972587:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760795.972740:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495965952 00000100:00100000:3.0:1789760795.972749:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495965952 00000100:00100000:3.0:1789760795.972808:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780198016242 time 780238957277 state 0x0 ref 1 00000100:00100000:3.0:1789760795.972824:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780239217693 time 780238957277 state 0x1 ref 1 00000100:00100000:3.0:1789760795.972855:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780239217693 time 780293571970 state 0x0 ref 1 00000100:00100000:3.0:1789760795.972868:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02fdca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495965952:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760795.972915:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28205056, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd809 00010000:00010000:3.0:1789760795.972925:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548416 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.972946:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760795.972951:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.973161:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548416 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28205056->28209151] (req 28205056->28209151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.973179:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548416 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28205056->28209151] (req 28205056->28209151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.999652:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974203 is committed 00002000:00010000:3.0:1789760795.999813:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28205056 -> 28209152 00010000:00010000:3.0:1789760795.999829:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548416 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28205056->28209151] (req 28205056->28209151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.999843:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548416 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28205056->28209151] (req 28205056->28209151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.999852:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548416 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28205056->28209151] (req 28205056->28209151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.999859:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760795.999864:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548416 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28205056->28209151] (req 28205056->28209151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.999874:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548416 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28205056->28209151] (req 28205056->28209151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.999895:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548416 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28205056->28209151] (req 28205056->28209151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.999904:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548416 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28205056->28209151] (req 28205056->28209151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.000102:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02fdca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495965952:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27230us (27522us total) trans 4294974203 rc 0/0 00000100:00100000:3.0:1789760796.000114:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6915 00000100:00100000:3.0:1789760796.000126:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780239217693 time 780293571970 state 0x2 ref 0 00000100:00100000:1.0:1789760796.005815:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760796.005918:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495966080 00000100:00100000:3.0:1789760796.005928:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495966080 00000100:00100000:3.0:1789760796.005998:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780239217693 time 780293571970 state 0x0 ref 1 00000100:00100000:3.0:1789760796.006016:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780293832386 time 780293571970 state 0x1 ref 1 00000100:00100000:3.0:1789760796.006048:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780293832386 time 780326765141 state 0x0 ref 1 00000100:00100000:3.0:1789760796.006061:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495966080:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760796.006118:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28209152, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd809 00010000:00010000:3.0:1789760796.006128:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454841d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.006145:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760796.006150:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.006156:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454841d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28209152->28213247] (req 28209152->28213247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.006176:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454841d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28209152->28213247] (req 28209152->28213247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.036432:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974204 is committed 00002000:00010000:3.0:1789760796.036513:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28209152 -> 28213248 00010000:00010000:3.0:1789760796.036526:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454841d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28209152->28213247] (req 28209152->28213247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.036536:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454841d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28209152->28213247] (req 28209152->28213247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.036542:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454841d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28209152->28213247] (req 28209152->28213247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.036547:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760796.036550:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454841d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28209152->28213247] (req 28209152->28213247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.036557:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454841d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28209152->28213247] (req 28209152->28213247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.036573:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454841d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28209152->28213247] (req 28209152->28213247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.036577:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454841d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28209152->28213247] (req 28209152->28213247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.036770:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495966080:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30706us (30961us total) trans 4294974204 rc 0/0 00000100:00100000:3.0:1789760796.036782:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6916 00000100:00100000:3.0:1789760796.036792:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780293832386 time 780326765141 state 0x2 ref 0 00000100:00100000:0.0:1789760796.039844:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760796.040024:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495966208 00000100:00100000:1.0:1789760796.040033:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495966208 00000100:00100000:1.0:1789760796.040086:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780293832386 time 780326765141 state 0x0 ref 1 00000100:00100000:1.0:1789760796.040102:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780327025557 time 780326765141 state 0x1 ref 1 00000100:00100000:1.0:1789760796.040138:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780327025557 time 780360854384 state 0x0 ref 1 00000100:00100000:1.0:1789760796.040150:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495966208:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760796.040197:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28213248, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd809 00010000:00010000:1.0:1789760796.040207:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354548424 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.040229:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760796.040234:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.040241:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354548424 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28213248->28217343] (req 28213248->28217343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.040256:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354548424 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28213248->28217343] (req 28213248->28217343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.070836:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974205 is committed 00002000:00010000:0.0:1789760796.070936:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28213248 -> 28217344 00010000:00010000:0.0:1789760796.070951:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354548424 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28213248->28217343] (req 28213248->28217343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.071038:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354548424 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28213248->28217343] (req 28213248->28217343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.071049:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354548424 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28213248->28217343] (req 28213248->28217343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.071057:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760796.071062:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354548424 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28213248->28217343] (req 28213248->28217343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.071072:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354548424 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28213248->28217343] (req 28213248->28217343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.071091:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354548424 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28213248->28217343] (req 28213248->28217343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.071100:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047f9b8c5/0x4936748354548424 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28213248->28217343] (req 28213248->28217343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.071192:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495966208:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31040us (31352us total) trans 4294974205 rc 0/0 00000100:00100000:0.0:1789760796.071204:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6917 00000100:00100000:0.0:1789760796.071214:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780327025557 time 780360854384 state 0x2 ref 0 00000100:00100000:2.0:1789760796.075338:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760796.075460:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495966336 00000100:00100000:3.0:1789760796.075470:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495966336 00000100:00100000:3.0:1789760796.075528:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780327025557 time 780360854384 state 0x0 ref 1 00000100:00100000:3.0:1789760796.075543:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780361114800 time 780360854384 state 0x1 ref 1 00000100:00100000:3.0:1789760796.075570:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780361114800 time 780396286689 state 0x0 ref 1 00000100:00100000:3.0:1789760796.075582:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495966336:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760796.075648:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28217344, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd809 00010000:00010000:3.0:1789760796.075657:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454842b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.075673:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760796.075677:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.075684:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454842b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28217344->28221439] (req 28217344->28221439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.075697:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454842b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28217344->28221439] (req 28217344->28221439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760796.105995:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28217344 -> 28221440 00010000:00010000:3.0:1789760796.106019:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454842b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28217344->28221439] (req 28217344->28221439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.106034:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454842b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28217344->28221439] (req 28217344->28221439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.106045:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454842b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28217344->28221439] (req 28217344->28221439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.106052:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760796.106058:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454842b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28217344->28221439] (req 28217344->28221439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.106067:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454842b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28217344->28221439] (req 28217344->28221439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.106090:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454842b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28217344->28221439] (req 28217344->28221439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.106098:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x493674835454842b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28217344->28221439] (req 28217344->28221439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.106211:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495966336:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30626us (30878us total) trans 4294974206 rc 0/0 00000100:00100000:3.0:1789760796.106226:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6918 00000100:00100000:3.0:1789760796.106237:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780361114800 time 780396286689 state 0x2 ref 0 00000001:00080000:0.0:1789760796.107755:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974206 is committed 00000100:00100000:2.0:1789760796.118846:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760796.122104:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495966464 00000100:00100000:3.0:1789760796.122120:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495966464 00000100:00100000:3.0:1789760796.122185:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780361114800 time 780396286689 state 0x0 ref 1 00000100:00100000:3.0:1789760796.122201:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780396547105 time 780396286689 state 0x1 ref 1 00000100:00100000:3.0:1789760796.122326:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780396547105 time 780443041869 state 0x0 ref 1 00000100:00100000:3.0:1789760796.122341:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02768e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495966464:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760796.122387:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28221440, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd809 00010000:00010000:3.0:1789760796.122396:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548432 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.122412:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760796.122417:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.122423:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548432 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28221440->28225535] (req 28221440->28225535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.122436:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548432 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28221440->28225535] (req 28221440->28225535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.161424:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974207 is committed 00002000:00010000:3.0:1789760796.161622:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28221440 -> 28225536 00010000:00010000:3.0:1789760796.161647:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548432 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28221440->28225535] (req 28221440->28225535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.161662:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548432 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28221440->28225535] (req 28221440->28225535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.161675:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548432 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28221440->28225535] (req 28221440->28225535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.161683:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760796.161687:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548432 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28221440->28225535] (req 28221440->28225535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.161697:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548432 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28221440->28225535] (req 28221440->28225535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.161716:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548432 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28221440->28225535] (req 28221440->28225535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.161724:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548432 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28221440->28225535] (req 28221440->28225535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.161804:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02768e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495966464:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39462us (42964us total) trans 4294974207 rc 0/0 00000100:00100000:3.0:1789760796.161814:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6919 00000100:00100000:3.0:1789760796.161825:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780396547105 time 780443041869 state 0x2 ref 0 00000100:00100000:2.0:1789760796.164521:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760796.164645:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495966592 00000100:00100000:0.0:1789760796.164655:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495966592 00000100:00100000:0.0:1789760796.164860:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780396547105 time 780443041869 state 0x0 ref 1 00000100:00100000:0.0:1789760796.164876:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780443302285 time 780443041869 state 0x1 ref 1 00000100:00100000:0.0:1789760796.164906:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780443302285 time 780485623232 state 0x0 ref 1 00000100:00100000:0.0:1789760796.164917:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495966592:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760796.164961:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28225536, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd809 00010000:00010000:0.0:1789760796.164969:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354548439 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.164986:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=780s v=(0 0) 00010000:00010000:0.0:1789760796.164991:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.164997:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354548439 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28225536->28229631] (req 28225536->28229631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.165016:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354548439 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28225536->28229631] (req 28225536->28229631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.210172:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974208 is committed 00002000:00010000:1.0:1789760796.210315:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28225536 -> 28229632 00010000:00010000:1.0:1789760796.210331:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354548439 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28225536->28229631] (req 28225536->28229631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.210343:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354548439 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28225536->28229631] (req 28225536->28229631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.210352:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354548439 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28225536->28229631] (req 28225536->28229631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.210359:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760796.210364:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354548439 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28225536->28229631] (req 28225536->28229631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.210373:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354548439 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28225536->28229631] (req 28225536->28229631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.210391:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354548439 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28225536->28229631] (req 28225536->28229631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.210399:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000392302ee/0x4936748354548439 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28225536->28229631] (req 28225536->28229631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.210501:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495966592:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45581us (45984us total) trans 4294974208 rc 0/0 00000100:00100000:1.0:1789760796.210512:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6920 00000100:00100000:1.0:1789760796.210523:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780443302285 time 780485623232 state 0x2 ref 0 00000100:00100000:2.0:1789760796.215174:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760796.215266:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495966720 00000100:00100000:3.0:1789760796.215276:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495966720 00000100:00100000:3.0:1789760796.215380:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780443302285 time 780485623232 state 0x0 ref 1 00000100:00100000:3.0:1789760796.215396:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780485883648 time 780485623232 state 0x1 ref 1 00000100:00100000:3.0:1789760796.215423:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780485883648 time 780536140392 state 0x0 ref 1 00000100:00100000:3.0:1789760796.215434:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495966720:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760796.215481:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28229632, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd809 00010000:00010000:3.0:1789760796.215490:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548440 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.215505:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760796.215511:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.215517:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548440 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28229632->28233727] (req 28229632->28233727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.215532:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548440 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28229632->28233727] (req 28229632->28233727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760796.248984:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28229632 -> 28233728 00010000:00010000:3.0:1789760796.249005:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548440 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28229632->28233727] (req 28229632->28233727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.249018:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548440 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28229632->28233727] (req 28229632->28233727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.249033:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548440 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28229632->28233727] (req 28229632->28233727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.249041:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760796.249046:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548440 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28229632->28233727] (req 28229632->28233727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.249054:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548440 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28229632->28233727] (req 28229632->28233727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.249072:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548440 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28229632->28233727] (req 28229632->28233727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.249081:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548440 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28229632->28233727] (req 28229632->28233727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.249170:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495966720:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33733us (34004us total) trans 4294974209 rc 0/0 00000100:00100000:3.0:1789760796.249181:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6921 00000100:00100000:3.0:1789760796.249191:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780485883648 time 780536140392 state 0x2 ref 0 00000001:00080000:2.0:1789760796.249708:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974209 is committed 00000100:00100000:2.0:1789760796.253962:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760796.254197:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495966848 00000100:00100000:3.0:1789760796.254212:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495966848 00000100:00100000:3.0:1789760796.254270:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780485883648 time 780536140392 state 0x0 ref 1 00000100:00100000:3.0:1789760796.254362:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780536400808 time 780536140392 state 0x1 ref 1 00000100:00100000:3.0:1789760796.254396:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780536400808 time 780575112827 state 0x0 ref 1 00000100:00100000:3.0:1789760796.254410:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fba3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495966848:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760796.254458:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28233728, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd809 00010000:00010000:3.0:1789760796.254466:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548447 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.254483:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760796.254488:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.254495:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548447 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28233728->28237823] (req 28233728->28237823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.254512:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548447 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28233728->28237823] (req 28233728->28237823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.279443:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974210 is committed 00002000:00010000:3.0:1789760796.279727:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28233728 -> 28237824 00010000:00010000:3.0:1789760796.279742:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548447 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28233728->28237823] (req 28233728->28237823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.279756:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548447 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28233728->28237823] (req 28233728->28237823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.279765:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548447 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28233728->28237823] (req 28233728->28237823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.279773:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760796.279778:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548447 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28233728->28237823] (req 28233728->28237823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.279787:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548447 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28233728->28237823] (req 28233728->28237823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.279807:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548447 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28233728->28237823] (req 28233728->28237823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.279815:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548447 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28233728->28237823] (req 28233728->28237823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.279905:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fba3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495966848:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25494us (25949us total) trans 4294974210 rc 0/0 00000100:00100000:3.0:1789760796.279915:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6922 00000100:00100000:3.0:1789760796.279925:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780536400808 time 780575112827 state 0x2 ref 0 00000100:00100000:1.0:1789760796.291970:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760796.292074:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495966976 00000100:00100000:0.0:1789760796.292083:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495966976 00000100:00100000:0.0:1789760796.292137:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780536400808 time 780575112827 state 0x0 ref 1 00000100:00100000:0.0:1789760796.292152:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780575373243 time 780575112827 state 0x1 ref 1 00000100:00100000:0.0:1789760796.292190:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780575373243 time 780612906217 state 0x0 ref 1 00000100:00100000:0.0:1789760796.292203:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495966976:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760796.292253:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28237824, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd809 00010000:00010000:0.0:1789760796.292262:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454844e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.292314:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=780s v=(0 0) 00010000:00010000:0.0:1789760796.292320:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.292327:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454844e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28237824->28241919] (req 28237824->28241919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.292342:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454844e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28237824->28241919] (req 28237824->28241919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.316041:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974211 is committed 00002000:00010000:0.0:1789760796.316131:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28237824 -> 28241920 00010000:00010000:0.0:1789760796.316146:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454844e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28237824->28241919] (req 28237824->28241919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.316160:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454844e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28237824->28241919] (req 28237824->28241919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.316174:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454844e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28237824->28241919] (req 28237824->28241919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.316182:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760796.316187:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454844e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28237824->28241919] (req 28237824->28241919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.316196:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454844e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28237824->28241919] (req 28237824->28241919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.316215:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454844e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28237824->28241919] (req 28237824->28241919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.316224:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454844e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28237824->28241919] (req 28237824->28241919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.316339:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495966976:12345-192.168.203.9@tcp:4:dd.0 Request processed in 24135us (24374us total) trans 4294974211 rc 0/0 00000100:00100000:0.0:1789760796.316349:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6923 00000100:00100000:0.0:1789760796.316358:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780575373243 time 780612906217 state 0x2 ref 0 00000100:00100000:2.0:1789760796.321867:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760796.321980:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495967104 00000100:00100000:3.0:1789760796.321990:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495967104 00000100:00100000:3.0:1789760796.322046:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780575373243 time 780612906217 state 0x0 ref 1 00000100:00100000:3.0:1789760796.322063:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780613166633 time 780612906217 state 0x1 ref 1 00000100:00100000:3.0:1789760796.322103:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780613166633 time 780642819128 state 0x0 ref 1 00000100:00100000:3.0:1789760796.322116:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495967104:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760796.322165:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28241920, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd809 00010000:00010000:3.0:1789760796.322175:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548455 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.322192:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760796.322197:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.322203:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548455 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28241920->28246015] (req 28241920->28246015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.322219:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548455 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28241920->28246015] (req 28241920->28246015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.378432:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974212 is committed 00002000:00010000:1.0:1789760796.378783:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28241920 -> 28246016 00010000:00010000:1.0:1789760796.378800:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548455 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28241920->28246015] (req 28241920->28246015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.378812:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548455 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28241920->28246015] (req 28241920->28246015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.378822:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548455 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28241920->28246015] (req 28241920->28246015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.378829:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760796.378834:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548455 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28241920->28246015] (req 28241920->28246015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.378843:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548455 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28241920->28246015] (req 28241920->28246015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.378862:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548455 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28241920->28246015] (req 28241920->28246015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.378871:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548455 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28241920->28246015] (req 28241920->28246015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.379266:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495967104:12345-192.168.203.9@tcp:4:dd.0 Request processed in 57147us (57403us total) trans 4294974212 rc 0/0 00000100:00100000:1.0:1789760796.379327:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6924 00000100:00100000:1.0:1789760796.379339:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780613166633 time 780642819128 state 0x2 ref 0 00000100:00100000:2.0:1789760796.387549:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760796.387880:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495967232 00000100:00100000:3.0:1789760796.387892:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495967232 00000100:00100000:3.0:1789760796.387947:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780613166633 time 780642819128 state 0x0 ref 1 00000100:00100000:3.0:1789760796.387972:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780643079544 time 780642819128 state 0x1 ref 1 00000100:00100000:3.0:1789760796.388046:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780643079544 time 780708762070 state 0x0 ref 1 00000100:00100000:3.0:1789760796.388059:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02768e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495967232:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760796.388122:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28246016, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd809 00010000:00010000:3.0:1789760796.388131:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454845c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.388146:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760796.388150:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.388157:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454845c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28246016->28250111] (req 28246016->28250111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.388171:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454845c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28246016->28250111] (req 28246016->28250111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760796.429504:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28246016 -> 28250112 00010000:00010000:3.0:1789760796.429527:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454845c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28246016->28250111] (req 28246016->28250111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.429541:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454845c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28246016->28250111] (req 28246016->28250111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.429551:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454845c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28246016->28250111] (req 28246016->28250111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.429559:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760796.429564:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454845c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28246016->28250111] (req 28246016->28250111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.429574:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454845c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28246016->28250111] (req 28246016->28250111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.429594:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454845c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28246016->28250111] (req 28246016->28250111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.429604:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x493674835454845c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28246016->28250111] (req 28246016->28250111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.429709:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02768e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495967232:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41647us (42163us total) trans 4294974213 rc 0/0 00000100:00100000:3.0:1789760796.429720:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6925 00000100:00100000:3.0:1789760796.429731:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780643079544 time 780708762070 state 0x2 ref 0 00000001:00080000:1.0:1789760796.430250:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974213 is committed 00000100:00100000:3.0:1789760796.433118:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760796.433435:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495967360 00000100:00100000:0.0:1789760796.433445:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495967360 00000100:00100000:1.0:1789760796.433605:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780643079544 time 780708762070 state 0x0 ref 1 00000100:00100000:1.0:1789760796.433833:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780709022486 time 780708762070 state 0x1 ref 1 00000100:00100000:1.0:1789760796.433912:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780709022486 time 780754626633 state 0x0 ref 1 00000100:00100000:1.0:1789760796.433926:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495967360:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760796.433986:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28250112, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd809 00010000:00010000:1.0:1789760796.433995:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354548463 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.434011:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760796.434015:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.434021:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354548463 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28250112->28254207] (req 28250112->28254207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.434035:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354548463 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28250112->28254207] (req 28250112->28254207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760796.472225:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28250112 -> 28254208 00010000:00010000:3.0:1789760796.472245:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354548463 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28250112->28254207] (req 28250112->28254207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.472258:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354548463 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28250112->28254207] (req 28250112->28254207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.472276:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354548463 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28250112->28254207] (req 28250112->28254207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.472298:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760796.472304:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354548463 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28250112->28254207] (req 28250112->28254207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.472313:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354548463 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28250112->28254207] (req 28250112->28254207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.472333:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354548463 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28250112->28254207] (req 28250112->28254207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.472347:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a50a4fd/0x4936748354548463 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28250112->28254207] (req 28250112->28254207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.472430:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495967360:12345-192.168.203.9@tcp:4:dd.0 Request processed in 38504us (39317us total) trans 4294974214 rc 0/0 00000100:00100000:3.0:1789760796.472439:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6926 00000100:00100000:3.0:1789760796.472449:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780709022486 time 780754626633 state 0x2 ref 0 00000001:00080000:0.0:1789760796.473501:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974214 is committed 00000100:00100000:2.0:1789760796.476522:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760796.476600:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495967488 00000100:00100000:1.0:1789760796.476618:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495967488 00000100:00100000:1.0:1789760796.476680:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780709022486 time 780754626633 state 0x0 ref 1 00000100:00100000:1.0:1789760796.476695:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780754887049 time 780754626633 state 0x1 ref 1 00000100:00100000:1.0:1789760796.476723:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780754887049 time 780797439910 state 0x0 ref 1 00000100:00100000:1.0:1789760796.476734:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f02769880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495967488:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760796.476780:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28254208, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd809 00010000:00010000:1.0:1789760796.476789:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454846a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.476804:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760796.476808:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.476815:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454846a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28254208->28258303] (req 28254208->28258303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.476828:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454846a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28254208->28258303] (req 28254208->28258303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760796.508235:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28254208 -> 28258304 00010000:00010000:1.0:1789760796.508255:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454846a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28254208->28258303] (req 28254208->28258303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.508275:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454846a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28254208->28258303] (req 28254208->28258303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.508312:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454846a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28254208->28258303] (req 28254208->28258303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.508320:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760796.508325:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454846a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28254208->28258303] (req 28254208->28258303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.508334:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454846a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28254208->28258303] (req 28254208->28258303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.508352:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454846a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28254208->28258303] (req 28254208->28258303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.508360:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x493674835454846a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28254208->28258303] (req 28254208->28258303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.508447:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f02769880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495967488:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31710us (31929us total) trans 4294974215 rc 0/0 00000100:00100000:1.0:1789760796.508458:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6927 00000100:00100000:1.0:1789760796.508468:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780754887049 time 780797439910 state 0x2 ref 0 00000001:00080000:3.0:1789760796.509193:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974215 is committed 00010000:00010000:1.0:1789760796.510472:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760796.515027:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760796.515140:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495967616 00000100:00100000:0.0:1789760796.515150:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495967616 00000100:00100000:0.0:1789760796.515208:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780754887049 time 780797439910 state 0x0 ref 1 00000100:00100000:0.0:1789760796.515224:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780797700326 time 780797439910 state 0x1 ref 1 00000100:00100000:0.0:1789760796.515398:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780797700326 time 780836112752 state 0x0 ref 1 00000100:00100000:0.0:1789760796.515412:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495967616:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760796.515466:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28258304, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd809 00010000:00010000:0.0:1789760796.515477:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354548471 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.515493:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=780s v=(0 0) 00010000:00010000:0.0:1789760796.515499:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.515506:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354548471 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28258304->28262399] (req 28258304->28262399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.515523:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354548471 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28258304->28262399] (req 28258304->28262399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.549204:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974216 is committed 00002000:00010000:0.0:1789760796.549242:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28258304 -> 28262400 00010000:00010000:0.0:1789760796.549257:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354548471 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28258304->28262399] (req 28258304->28262399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.549269:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354548471 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28258304->28262399] (req 28258304->28262399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.549300:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354548471 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28258304->28262399] (req 28258304->28262399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.549308:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94aa00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760796.549314:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354548471 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28258304->28262399] (req 28258304->28262399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.549323:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354548471 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28258304->28262399] (req 28258304->28262399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.549344:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354548471 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28258304->28262399] (req 28258304->28262399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.549352:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9f90d82/0x4936748354548471 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28258304->28262399] (req 28258304->28262399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.549442:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495967616:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34029us (34421us total) trans 4294974216 rc 0/0 00000100:00100000:0.0:1789760796.549452:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6928 00000100:00100000:0.0:1789760796.549461:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780797700326 time 780836112752 state 0x2 ref 0 00000100:00100000:0.0:1789760796.554229:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760796.554598:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495967744 00000100:00100000:3.0:1789760796.554607:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495967744 00000100:00100000:3.0:1789760796.554669:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780797700326 time 780836112752 state 0x0 ref 1 00000100:00100000:3.0:1789760796.554685:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780836373168 time 780836112752 state 0x1 ref 1 00000100:00100000:3.0:1789760796.554877:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780836373168 time 780875592366 state 0x0 ref 1 00000100:00100000:3.0:1789760796.554889:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495967744:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760796.554937:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28262400, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd809 00010000:00010000:3.0:1789760796.554946:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548478 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.554962:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760796.554967:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.554973:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548478 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28262400->28266495] (req 28262400->28266495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.554986:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548478 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28262400->28266495] (req 28262400->28266495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.599447:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974217 is committed 00002000:00010000:1.0:1789760796.599649:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28262400 -> 28266496 00010000:00010000:1.0:1789760796.599665:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548478 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28262400->28266495] (req 28262400->28266495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.599678:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548478 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28262400->28266495] (req 28262400->28266495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.599687:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548478 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28262400->28266495] (req 28262400->28266495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.599696:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760796.599700:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548478 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28262400->28266495] (req 28262400->28266495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.599709:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548478 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28262400->28266495] (req 28262400->28266495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.599727:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548478 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28262400->28266495] (req 28262400->28266495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.599735:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548478 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28262400->28266495] (req 28262400->28266495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.599824:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495967744:12345-192.168.203.9@tcp:4:dd.0 Request processed in 44932us (45600us total) trans 4294974217 rc 0/0 00000100:00100000:1.0:1789760796.599834:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6929 00000100:00100000:1.0:1789760796.599845:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780836373168 time 780875592366 state 0x2 ref 0 00000100:00100000:1.0:1789760796.604162:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760796.604451:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495967872 00000100:00100000:3.0:1789760796.604460:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495967872 00000100:00100000:3.0:1789760796.604512:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780836373168 time 780875592366 state 0x0 ref 1 00000100:00100000:3.0:1789760796.604527:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780875852782 time 780875592366 state 0x1 ref 1 00000100:00100000:3.0:1789760796.604551:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780875852782 time 780925268329 state 0x0 ref 1 00000100:00100000:3.0:1789760796.604563:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495967872:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760796.604616:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28266496, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd809 00010000:00010000:3.0:1789760796.604625:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454847f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.604652:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760796.604656:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.604662:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454847f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28266496->28270591] (req 28266496->28270591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.604675:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454847f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28266496->28270591] (req 28266496->28270591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.644441:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974218 is committed 00002000:00010000:1.0:1789760796.644545:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28266496 -> 28270592 00010000:00010000:1.0:1789760796.644560:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454847f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28266496->28270591] (req 28266496->28270591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.644573:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454847f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28266496->28270591] (req 28266496->28270591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.644582:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454847f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28266496->28270591] (req 28266496->28270591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.644589:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760796.644594:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454847f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28266496->28270591] (req 28266496->28270591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.644602:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454847f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28266496->28270591] (req 28266496->28270591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.644639:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454847f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28266496->28270591] (req 28266496->28270591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.644646:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x493674835454847f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28266496->28270591] (req 28266496->28270591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.644777:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495967872:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40211us (40618us total) trans 4294974218 rc 0/0 00000100:00100000:1.0:1789760796.644787:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6930 00000100:00100000:1.0:1789760796.644798:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780875852782 time 780925268329 state 0x2 ref 0 00000100:00100000:3.0:1789760796.648412:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760796.648543:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495968000 00000100:00100000:0.0:1789760796.648559:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495968000 00000100:00100000:0.0:1789760796.648647:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780875852782 time 780925268329 state 0x0 ref 1 00000100:00100000:0.0:1789760796.648671:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780925528745 time 780925268329 state 0x1 ref 1 00000100:00100000:0.0:1789760796.648717:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780925528745 time 780969431656 state 0x0 ref 1 00000100:00100000:0.0:1789760796.648737:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495968000:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760796.648792:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28270592, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd809 00010000:00010000:0.0:1789760796.648801:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354548486 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.648817:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=780s v=(0 0) 00010000:00010000:0.0:1789760796.648821:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.648827:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354548486 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28270592->28274687] (req 28270592->28274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.648842:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354548486 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28270592->28274687] (req 28270592->28274687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.673995:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974219 is committed 00002000:00010000:0.0:1789760796.674102:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28270592 -> 28274688 00002000:00010000:0.0:1789760796.674106:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760795 -> 1789760796 00002000:00010000:0.0:1789760796.674109:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760795 -> 1789760796 00010000:00010000:0.0:1789760796.674122:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354548486 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28270592->28274687] (req 28270592->28274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.674135:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354548486 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28270592->28274687] (req 28270592->28274687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.674143:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354548486 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28270592->28274687] (req 28270592->28274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.674151:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ae00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760796.674157:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354548486 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28270592->28274687] (req 28270592->28274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.674167:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354548486 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28270592->28274687] (req 28270592->28274687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.674188:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354548486 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28270592->28274687] (req 28270592->28274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.674196:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9ea27cb/0x4936748354548486 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28270592->28274687] (req 28270592->28274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.674324:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495968000:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25586us (25916us total) trans 4294974219 rc 0/0 00000100:00100000:0.0:1789760796.674334:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6931 00000100:00100000:0.0:1789760796.674345:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780925528745 time 780969431656 state 0x2 ref 0 00000100:00100000:0.0:1789760796.677968:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760796.678060:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495968128 00000100:00100000:1.0:1789760796.678070:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495968128 00000100:00100000:1.0:1789760796.678128:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780925528745 time 780969431656 state 0x0 ref 1 00000100:00100000:1.0:1789760796.678144:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780969692072 time 780969431656 state 0x1 ref 1 00000100:00100000:1.0:1789760796.678179:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780969692072 time 780998895981 state 0x0 ref 1 00000100:00100000:1.0:1789760796.678192:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900fab2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495968128:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760796.678243:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28274688, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd809 00010000:00010000:1.0:1789760796.678253:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454848d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.678269:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760796.678273:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.678317:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454848d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28274688->28278783] (req 28274688->28278783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.678333:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454848d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28274688->28278783] (req 28274688->28278783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.704190:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974220 is committed 00002000:00010000:1.0:1789760796.704230:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28274688 -> 28278784 00010000:00010000:1.0:1789760796.704408:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454848d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28274688->28278783] (req 28274688->28278783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.704422:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454848d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28274688->28278783] (req 28274688->28278783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.704431:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454848d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28274688->28278783] (req 28274688->28278783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.704438:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760796.704443:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454848d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28274688->28278783] (req 28274688->28278783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.704452:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454848d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28274688->28278783] (req 28274688->28278783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.704472:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454848d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28274688->28278783] (req 28274688->28278783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.704481:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e27b4abe/0x493674835454848d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28274688->28278783] (req 28274688->28278783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.704567:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900fab2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495968128:12345-192.168.203.9@tcp:4:dd.0 Request processed in 26373us (26603us total) trans 4294974220 rc 0/0 00000100:00100000:1.0:1789760796.704577:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6932 00000100:00100000:1.0:1789760796.704586:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780969692072 time 780998895981 state 0x2 ref 0 00000100:00100000:1.0:1789760796.707729:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760796.707843:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495968256 00000100:00100000:2.0:1789760796.707853:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495968256 00000100:00100000:2.0:1789760796.707909:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780969692072 time 780998895981 state 0x0 ref 1 00000100:00100000:2.0:1789760796.707924:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780999156397 time 780998895981 state 0x1 ref 1 00000100:00100000:2.0:1789760796.708065:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780999156397 time 781028780218 state 0x0 ref 1 00000100:00100000:2.0:1789760796.708079:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495968256:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760796.708128:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28278784, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd809 00010000:00010000:2.0:1789760796.708138:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548494 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.708153:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=781s v=(0 0) 00010000:00010000:2.0:1789760796.708157:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.708163:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548494 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28278784->28282879] (req 28278784->28282879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.708177:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548494 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28278784->28282879] (req 28278784->28282879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.747814:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974221 is committed 00002000:00010000:1.0:1789760796.747960:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28278784 -> 28282880 00010000:00010000:1.0:1789760796.747977:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548494 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28278784->28282879] (req 28278784->28282879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.747990:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548494 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28278784->28282879] (req 28278784->28282879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.748000:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548494 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28278784->28282879] (req 28278784->28282879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.748008:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fea00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760796.748013:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548494 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28278784->28282879] (req 28278784->28282879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.748023:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548494 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28278784->28282879] (req 28278784->28282879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.748043:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548494 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28278784->28282879] (req 28278784->28282879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.748052:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000543dfd03/0x4936748354548494 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28278784->28282879] (req 28278784->28282879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.748143:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495968256:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40061us (40683us total) trans 4294974221 rc 0/0 00000100:00100000:1.0:1789760796.748158:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6933 00000100:00100000:1.0:1789760796.748169:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780999156397 time 781028780218 state 0x2 ref 0 00000100:00100000:3.0:1789760796.757747:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760796.757934:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495968384 00000100:00100000:0.0:1789760796.757947:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495968384 00000100:00100000:0.0:1789760796.758007:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 780999156397 time 781028780218 state 0x0 ref 1 00000100:00100000:0.0:1789760796.758023:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781029040634 time 781028780218 state 0x1 ref 1 00000100:00100000:0.0:1789760796.758055:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781029040634 time 781078772431 state 0x0 ref 1 00000100:00100000:0.0:1789760796.758069:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495968384:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760796.758217:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28282880, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd809 00010000:00010000:0.0:1789760796.758227:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454849b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.758241:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=781s v=(0 0) 00010000:00010000:0.0:1789760796.758246:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.758252:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454849b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28282880->28286975] (req 28282880->28286975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.758265:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454849b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28282880->28286975] (req 28282880->28286975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760796.808133:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28282880 -> 28286976 00010000:00010000:3.0:1789760796.808152:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454849b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28282880->28286975] (req 28282880->28286975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.808166:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454849b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28282880->28286975] (req 28282880->28286975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.808175:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454849b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28282880->28286975] (req 28282880->28286975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.808181:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94be00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760796.808186:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454849b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28282880->28286975] (req 28282880->28286975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.808194:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454849b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28282880->28286975] (req 28282880->28286975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.808213:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454849b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28282880->28286975] (req 28282880->28286975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.808220:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8241a98/0x493674835454849b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28282880->28286975] (req 28282880->28286975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.808355:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495968384:12345-192.168.203.9@tcp:4:dd.0 Request processed in 50284us (50612us total) trans 4294974222 rc 0/0 00000100:00100000:3.0:1789760796.808366:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6934 00000100:00100000:3.0:1789760796.808377:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781029040634 time 781078772431 state 0x2 ref 0 00000001:00080000:1.0:1789760796.808577:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974222 is committed 00000100:00100000:0.0:1789760796.815381:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760796.815581:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495968512 00000100:00100000:1.0:1789760796.815591:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495968512 00000100:00100000:1.0:1789760796.815664:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781029040634 time 781078772431 state 0x0 ref 1 00000100:00100000:1.0:1789760796.815680:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781079032847 time 781078772431 state 0x1 ref 1 00000100:00100000:1.0:1789760796.815860:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781079032847 time 781136574961 state 0x0 ref 1 00000100:00100000:1.0:1789760796.815873:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495968512:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760796.815922:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28286976, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd809 00010000:00010000:1.0:1789760796.815931:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545484a2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.815947:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=781s v=(0 0) 00010000:00010000:1.0:1789760796.815952:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.815958:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545484a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28286976->28291071] (req 28286976->28291071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.815973:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545484a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28286976->28291071] (req 28286976->28291071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.861766:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974223 is committed 00002000:00010000:3.0:1789760796.861862:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28286976 -> 28291072 00010000:00010000:3.0:1789760796.861878:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545484a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28286976->28291071] (req 28286976->28291071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.861891:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545484a2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28286976->28291071] (req 28286976->28291071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.861900:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545484a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28286976->28291071] (req 28286976->28291071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.861906:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7e00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760796.861911:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545484a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28286976->28291071] (req 28286976->28291071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.861920:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545484a2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28286976->28291071] (req 28286976->28291071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.861940:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545484a2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28286976->28291071] (req 28286976->28291071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.861963:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dfb5df5/0x49367483545484a2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28286976->28291071] (req 28286976->28291071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.862050:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03593b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495968512:12345-192.168.203.9@tcp:4:dd.0 Request processed in 46175us (46777us total) trans 4294974223 rc 0/0 00000100:00100000:3.0:1789760796.862061:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6935 00000100:00100000:3.0:1789760796.862071:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781079032847 time 781136574961 state 0x2 ref 0 00000100:00100000:1.0:1789760796.867933:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760796.868068:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495968640 00000100:00100000:2.0:1789760796.868077:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495968640 00000100:00100000:2.0:1789760796.868135:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781079032847 time 781136574961 state 0x0 ref 1 00000100:00100000:2.0:1789760796.868151:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781136835377 time 781136574961 state 0x1 ref 1 00000100:00100000:2.0:1789760796.868187:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781136835377 time 781188902998 state 0x0 ref 1 00000100:00100000:2.0:1789760796.868198:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9290404a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495968640:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760796.868245:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28291072, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd809 00010000:00010000:2.0:1789760796.868255:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545484a9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.868507:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=781s v=(0 0) 00010000:00010000:2.0:1789760796.868513:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.868519:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545484a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28291072->28295167] (req 28291072->28295167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.868534:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545484a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28291072->28295167] (req 28291072->28295167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760796.913889:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28291072 -> 28295168 00010000:00010000:1.0:1789760796.913910:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545484a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28291072->28295167] (req 28291072->28295167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.913924:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545484a9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28291072->28295167] (req 28291072->28295167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.913933:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545484a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28291072->28295167] (req 28291072->28295167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.913941:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760796.913946:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545484a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28291072->28295167] (req 28291072->28295167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.913955:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545484a9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28291072->28295167] (req 28291072->28295167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.913975:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545484a9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28291072->28295167] (req 28291072->28295167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.913983:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e5b47d94/0x49367483545484a9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28291072->28295167] (req 28291072->28295167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.914083:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9290404a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495968640:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45881us (46155us total) trans 4294974224 rc 0/0 00000100:00100000:1.0:1789760796.914096:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6936 00000100:00100000:1.0:1789760796.914107:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781136835377 time 781188902998 state 0x2 ref 0 00000001:00080000:0.0:1789760796.914852:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974224 is committed 00000100:00100000:2.0:1789760796.924130:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760796.926830:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495968768 00000100:00100000:3.0:1789760796.926845:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495968768 00000100:00100000:3.0:1789760796.926917:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781136835377 time 781188902998 state 0x0 ref 1 00000100:00100000:3.0:1789760796.926933:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781189163414 time 781188902998 state 0x1 ref 1 00000100:00100000:3.0:1789760796.927095:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781189163414 time 781247809300 state 0x0 ref 1 00000100:00100000:3.0:1789760796.927110:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495968768:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760796.927168:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28295168, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd809 00010000:00010000:3.0:1789760796.927178:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484b0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.927195:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760796.927199:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.927206:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28295168->28299263] (req 28295168->28299263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.927224:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28295168->28299263] (req 28295168->28299263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.959763:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff929036f88a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520318208:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760796.959835:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760796.959907:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699520318208 00000100:00100000:3.0:1789760796.959914:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699520318208 00000100:00100000:3.0:1789760796.959978:0:6711:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 221 00000100:00100000:3.0:1789760796.959988:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520318208:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789760796.960011:0:6711:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527630336 pending 1703936 free 3830448128 avail 3824156672 00000100:00100000:3.0:1789760796.960055:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699520318208:12345-0@lo:13:osp-pre-0-0.0 Request processed in 68us (219us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760796.960062:0:6711:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 221 00000100:00080000:3.0:1789760796.960082:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760796.960118:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff929036f88a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3320:1876699520318208:0@lo:13:osp-pre-0-0.0 00000001:00080000:1.0:1789760796.978043:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974225 is committed 00002000:00010000:3.0:1789760796.978186:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28295168 -> 28299264 00010000:00010000:3.0:1789760796.978201:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28295168->28299263] (req 28295168->28299263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.978213:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484b0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28295168->28299263] (req 28295168->28299263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.978222:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28295168->28299263] (req 28295168->28299263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.978234:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760796.978240:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28295168->28299263] (req 28295168->28299263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.978249:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484b0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28295168->28299263] (req 28295168->28299263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.978267:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484b0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28295168->28299263] (req 28295168->28299263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.978357:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484b0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28295168->28299263] (req 28295168->28299263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.978451:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495968768:12345-192.168.203.9@tcp:4:dd.0 Request processed in 51340us (54327us total) trans 4294974225 rc 0/0 00000100:00100000:3.0:1789760796.978461:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6937 00000100:00100000:3.0:1789760796.978471:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781189163414 time 781247809300 state 0x2 ref 0 00000100:00100000:3.0:1789760796.981726:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760796.981852:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495968896 00000100:00100000:1.0:1789760796.981862:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495968896 00000100:00100000:1.0:1789760796.981917:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781189163414 time 781247809300 state 0x0 ref 1 00000100:00100000:1.0:1789760796.981932:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781248069716 time 781247809300 state 0x1 ref 1 00000100:00100000:1.0:1789760796.981961:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781248069716 time 781302677294 state 0x0 ref 1 00000100:00100000:1.0:1789760796.981974:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495968896:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760796.982021:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28299264, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd809 00010000:00010000:1.0:1789760796.982031:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545484b7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.982046:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=781s v=(0 0) 00010000:00010000:1.0:1789760796.982051:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.982057:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545484b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28299264->28303359] (req 28299264->28303359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.982071:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545484b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28299264->28303359] (req 28299264->28303359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.035562:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974226 is committed 00002000:00010000:0.0:1789760797.035696:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28299264 -> 28303360 00010000:00010000:0.0:1789760797.035712:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545484b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28299264->28303359] (req 28299264->28303359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.035726:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545484b7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28299264->28303359] (req 28299264->28303359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.035735:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545484b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28299264->28303359] (req 28299264->28303359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.035743:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760797.035749:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545484b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28299264->28303359] (req 28299264->28303359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.035758:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545484b7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28299264->28303359] (req 28299264->28303359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.035778:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545484b7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28299264->28303359] (req 28299264->28303359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.035786:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x49367483545484b7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28299264->28303359] (req 28299264->28303359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.035879:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929036f88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495968896:12345-192.168.203.9@tcp:4:dd.0 Request processed in 53902us (54157us total) trans 4294974226 rc 0/0 00000100:00100000:0.0:1789760797.035889:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6938 00000100:00100000:0.0:1789760797.035899:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781248069716 time 781302677294 state 0x2 ref 0 00000100:00100000:1.0:1789760797.040984:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760797.041117:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495969024 00000100:00100000:3.0:1789760797.041127:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495969024 00000100:00100000:3.0:1789760797.041196:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781248069716 time 781302677294 state 0x0 ref 1 00000100:00100000:3.0:1789760797.041212:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781302937710 time 781302677294 state 0x1 ref 1 00000100:00100000:3.0:1789760797.041248:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781302937710 time 781361964907 state 0x0 ref 1 00000100:00100000:3.0:1789760797.041261:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495969024:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760797.041343:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28303360, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd809 00010000:00010000:3.0:1789760797.041352:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484be lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.041368:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760797.041373:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.041380:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28303360->28307455] (req 28303360->28307455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.041394:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28303360->28307455] (req 28303360->28307455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760797.041700:0:5849:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760797.066643:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974227 is committed 00002000:00010000:3.0:1789760797.066891:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28303360 -> 28307456 00010000:00010000:3.0:1789760797.066908:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28303360->28307455] (req 28303360->28307455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.066922:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484be lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28303360->28307455] (req 28303360->28307455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.066932:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28303360->28307455] (req 28303360->28307455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.066940:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760797.066956:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28303360->28307455] (req 28303360->28307455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.066966:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484be lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28303360->28307455] (req 28303360->28307455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.066987:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484be lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28303360->28307455] (req 28303360->28307455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.066995:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484be lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28303360->28307455] (req 28303360->28307455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.067085:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04293b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495969024:12345-192.168.203.9@tcp:4:dd.0 Request processed in 25821us (26105us total) trans 4294974227 rc 0/0 00000100:00100000:3.0:1789760797.067095:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6939 00000100:00100000:3.0:1789760797.067105:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781302937710 time 781361964907 state 0x2 ref 0 00000100:00100000:3.0:1789760797.078442:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760797.078559:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495969152 00000100:00100000:0.0:1789760797.078568:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495969152 00000100:00100000:0.0:1789760797.078639:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781302937710 time 781361964907 state 0x0 ref 1 00000100:00100000:0.0:1789760797.078653:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781362225323 time 781361964907 state 0x1 ref 1 00000100:00100000:0.0:1789760797.078683:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781362225323 time 781399399718 state 0x0 ref 1 00000100:00100000:0.0:1789760797.078696:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495969152:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760797.078751:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28307456, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd809 00010000:00010000:0.0:1789760797.078760:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545484c5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.078776:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=781s v=(0 0) 00010000:00010000:0.0:1789760797.078780:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.078786:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545484c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28307456->28311551] (req 28307456->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.078798:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545484c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28307456->28311551] (req 28307456->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.126073:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974228 is committed 00002000:00010000:0.0:1789760797.126266:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28307456 -> 28311552 00010000:00010000:0.0:1789760797.126370:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545484c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28307456->28311551] (req 28307456->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.126385:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545484c5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28307456->28311551] (req 28307456->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.126394:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545484c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28307456->28311551] (req 28307456->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.126402:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760797.126406:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545484c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28307456->28311551] (req 28307456->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.126416:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545484c5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28307456->28311551] (req 28307456->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.126436:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545484c5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28307456->28311551] (req 28307456->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.126444:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a705f496/0x49367483545484c5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28307456->28311551] (req 28307456->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.126534:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495969152:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47835us (48096us total) trans 4294974228 rc 0/0 00000100:00100000:0.0:1789760797.126544:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6940 00000100:00100000:0.0:1789760797.126554:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781362225323 time 781399399718 state 0x2 ref 0 00000100:00100000:0.0:1789760797.131520:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760797.131596:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495969280 00000100:00100000:1.0:1789760797.131707:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495969280 00000100:00100000:1.0:1789760797.131764:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781362225323 time 781399399718 state 0x0 ref 1 00000100:00100000:1.0:1789760797.131779:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781399660134 time 781399399718 state 0x1 ref 1 00000100:00100000:1.0:1789760797.131810:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781399660134 time 781452526760 state 0x0 ref 1 00000100:00100000:1.0:1789760797.131823:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495969280:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760797.131875:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28311552, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd809 00010000:00010000:1.0:1789760797.131886:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484cc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.131902:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=781s v=(0 0) 00010000:00010000:1.0:1789760797.131907:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.131914:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28311552->28315647] (req 28311552->28315647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.131927:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28311552->28315647] (req 28311552->28315647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.165742:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974229 is committed 00002000:00010000:1.0:1789760797.165842:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28311552 -> 28315648 00002000:00010000:1.0:1789760797.165847:0:6714:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 55305 -> 57353 00010000:00010000:1.0:1789760797.165860:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28311552->28315647] (req 28311552->28315647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.165871:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484cc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28311552->28315647] (req 28311552->28315647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.165884:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28311552->28315647] (req 28311552->28315647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.165891:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760797.165896:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28311552->28315647] (req 28311552->28315647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.165904:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484cc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28311552->28315647] (req 28311552->28315647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.165924:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484cc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28311552->28315647] (req 28311552->28315647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.165931:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484cc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28311552->28315647] (req 28311552->28315647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.166026:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03592680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495969280:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34201us (34508us total) trans 4294974229 rc 0/0 00000100:00100000:1.0:1789760797.166036:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6941 00000100:00100000:1.0:1789760797.166046:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781399660134 time 781452526760 state 0x2 ref 0 00000100:00100000:1.0:1789760797.172098:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760797.172265:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495969408 00000100:00100000:3.0:1789760797.172276:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495969408 00000100:00100000:3.0:1789760797.172543:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781399660134 time 781452526760 state 0x0 ref 1 00000100:00100000:3.0:1789760797.172558:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781452787176 time 781452526760 state 0x1 ref 1 00000100:00100000:3.0:1789760797.172679:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781452787176 time 781493394454 state 0x0 ref 1 00000100:00100000:3.0:1789760797.172693:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04931f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495969408:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760797.172743:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28315648, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:3.0:1789760797.172752:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484d3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.172768:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760797.172773:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.172778:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28315648->28319743] (req 28315648->28319743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.172792:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28315648->28319743] (req 28315648->28319743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760797.200953:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28315648 -> 28319744 00010000:00010000:3.0:1789760797.200973:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28315648->28319743] (req 28315648->28319743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.200986:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484d3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28315648->28319743] (req 28315648->28319743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.201009:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28315648->28319743] (req 28315648->28319743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.201017:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760797.201021:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28315648->28319743] (req 28315648->28319743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.201031:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484d3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28315648->28319743] (req 28315648->28319743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.201051:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484d3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28315648->28319743] (req 28315648->28319743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.201059:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484d3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28315648->28319743] (req 28315648->28319743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.201221:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495969408:12345-192.168.203.9@tcp:4:dd.0 Request processed in 28521us (29121us total) trans 4294974230 rc 0/0 00000100:00100000:3.0:1789760797.201233:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6942 00000100:00100000:3.0:1789760797.201245:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781452787176 time 781493394454 state 0x2 ref 0 00000001:00080000:3.0:1789760797.201339:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974230 is committed 00000100:00100000:2.0:1789760797.207260:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760797.207406:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495969536 00000100:00100000:0.0:1789760797.207414:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495969536 00000100:00100000:0.0:1789760797.207462:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781452787176 time 781493394454 state 0x0 ref 1 00000100:00100000:0.0:1789760797.207477:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781493654870 time 781493394454 state 0x1 ref 1 00000100:00100000:0.0:1789760797.207503:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781493654870 time 781528220739 state 0x0 ref 1 00000100:00100000:0.0:1789760797.207514:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495969536:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760797.207556:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28319744, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:0.0:1789760797.207565:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545484da lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.207588:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=781s v=(0 0) 00010000:00010000:0.0:1789760797.207592:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.207598:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545484da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28319744->28323839] (req 28319744->28323839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.207618:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545484da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28319744->28323839] (req 28319744->28323839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760797.275202:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28319744 -> 28323840 00010000:00010000:3.0:1789760797.275221:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545484da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28319744->28323839] (req 28319744->28323839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.275236:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545484da lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28319744->28323839] (req 28319744->28323839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.275245:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545484da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28319744->28323839] (req 28319744->28323839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.275253:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94ac00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760797.275257:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545484da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28319744->28323839] (req 28319744->28323839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.275273:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545484da lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28319744->28323839] (req 28319744->28323839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.275329:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545484da lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28319744->28323839] (req 28319744->28323839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.275338:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009acdbdb7/0x49367483545484da lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28319744->28323839] (req 28319744->28323839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.275437:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0492d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495969536:12345-192.168.203.9@tcp:4:dd.0 Request processed in 67919us (68180us total) trans 4294974231 rc 0/0 00000100:00100000:3.0:1789760797.275447:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6943 00000100:00100000:3.0:1789760797.275458:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781493654870 time 781528220739 state 0x2 ref 0 00000001:00080000:2.0:1789760797.279221:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974231 is committed 00000100:00100000:0.0:1789760797.280919:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760797.281081:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495969664 00000100:00100000:1.0:1789760797.281090:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495969664 00000100:00100000:1.0:1789760797.281142:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781493654870 time 781528220739 state 0x0 ref 1 00000100:00100000:1.0:1789760797.281162:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781528481155 time 781528220739 state 0x1 ref 1 00000100:00100000:1.0:1789760797.281227:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781528481155 time 781601943375 state 0x0 ref 1 00000100:00100000:1.0:1789760797.281239:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495969664:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760797.281311:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28323840, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:1.0:1789760797.281319:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484e1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.281334:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=781s v=(0 0) 00010000:00010000:1.0:1789760797.281338:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.281344:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28323840->28327935] (req 28323840->28327935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.281356:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28323840->28327935] (req 28323840->28327935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760797.320837:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28323840 -> 28327936 00010000:00010000:1.0:1789760797.320856:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28323840->28327935] (req 28323840->28327935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.320869:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484e1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28323840->28327935] (req 28323840->28327935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.320879:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28323840->28327935] (req 28323840->28327935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.320891:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760797.320896:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28323840->28327935] (req 28323840->28327935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.320904:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484e1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28323840->28327935] (req 28323840->28327935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.320921:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484e1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28323840->28327935] (req 28323840->28327935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.320927:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484e1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28323840->28327935] (req 28323840->28327935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.321125:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03590380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495969664:12345-192.168.203.9@tcp:4:dd.0 Request processed in 39882us (40209us total) trans 4294974232 rc 0/0 00000100:00100000:1.0:1789760797.321136:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6944 00000100:00100000:1.0:1789760797.321146:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781528481155 time 781601943375 state 0x2 ref 0 00000001:00080000:2.0:1789760797.322246:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974232 is committed 00000100:00100000:0.0:1789760797.326686:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760797.326785:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495969792 00000100:00100000:3.0:1789760797.326795:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495969792 00000100:00100000:3.0:1789760797.326854:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781528481155 time 781601943375 state 0x0 ref 1 00000100:00100000:3.0:1789760797.326868:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781602203791 time 781601943375 state 0x1 ref 1 00000100:00100000:3.0:1789760797.326956:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781602203791 time 781647672688 state 0x0 ref 1 00000100:00100000:3.0:1789760797.326970:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903fd0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495969792:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760797.327017:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28327936, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:3.0:1789760797.327027:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484e8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.327043:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760797.327048:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.327054:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28327936->28332031] (req 28327936->28332031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.327067:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28327936->28332031] (req 28327936->28332031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.356597:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974233 is committed 00002000:00010000:1.0:1789760797.356769:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28327936 -> 28332032 00010000:00010000:1.0:1789760797.356785:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28327936->28332031] (req 28327936->28332031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.356798:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484e8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28327936->28332031] (req 28327936->28332031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.356811:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28327936->28332031] (req 28327936->28332031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.356818:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760797.356823:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28327936->28332031] (req 28327936->28332031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.356831:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484e8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28327936->28332031] (req 28327936->28332031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.356849:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484e8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28327936->28332031] (req 28327936->28332031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.356857:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x49367483545484e8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28327936->28332031] (req 28327936->28332031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.357021:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903fd0ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495969792:12345-192.168.203.9@tcp:4:dd.0 Request processed in 29975us (30265us total) trans 4294974233 rc 0/0 00000100:00100000:1.0:1789760797.357032:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6945 00000100:00100000:1.0:1789760797.357042:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781602203791 time 781647672688 state 0x2 ref 0 00000100:00100000:2.0:1789760797.362692:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760797.362801:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495969920 00000100:00100000:3.0:1789760797.362811:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495969920 00000100:00100000:3.0:1789760797.362863:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781602203791 time 781647672688 state 0x0 ref 1 00000100:00100000:3.0:1789760797.362878:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781647933104 time 781647672688 state 0x1 ref 1 00000100:00100000:3.0:1789760797.362943:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781647933104 time 781683660055 state 0x0 ref 1 00000100:00100000:3.0:1789760797.363028:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495969920:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760797.363075:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28332032, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:3.0:1789760797.363084:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484ef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.363100:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760797.363105:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.363110:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28332032->28336127] (req 28332032->28336127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.363130:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28332032->28336127] (req 28332032->28336127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760797.395908:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28332032 -> 28336128 00010000:00010000:2.0:1789760797.395927:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28332032->28336127] (req 28332032->28336127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.395939:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484ef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28332032->28336127] (req 28332032->28336127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.396025:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28332032->28336127] (req 28332032->28336127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.396034:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760797.396038:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28332032->28336127] (req 28332032->28336127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.396049:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28332032->28336127] (req 28332032->28336127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.396069:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484ef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28332032->28336127] (req 28332032->28336127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.396077:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484ef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28332032->28336127] (req 28332032->28336127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.396173:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2358:x1876699495969920:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33143us (33485us total) trans 4294974234 rc 0/0 00000100:00100000:2.0:1789760797.396187:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6946 00000100:00100000:2.0:1789760797.396197:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781647933104 time 781683660055 state 0x2 ref 0 00000001:00080000:2.0:1789760797.396242:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974234 is committed 00000100:00100000:0.0:1789760797.400502:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760797.400614:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495970048 00000100:00100000:1.0:1789760797.400633:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495970048 00000100:00100000:1.0:1789760797.400688:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781647933104 time 781683660055 state 0x0 ref 1 00000100:00100000:1.0:1789760797.400704:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781683920471 time 781683660055 state 0x1 ref 1 00000100:00100000:1.0:1789760797.400735:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781683920471 time 781721451617 state 0x0 ref 1 00000100:00100000:1.0:1789760797.400747:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495970048:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760797.400799:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28336128, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:1.0:1789760797.400807:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484f6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.400822:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=781s v=(0 0) 00010000:00010000:1.0:1789760797.400827:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.400832:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28336128->28340223] (req 28336128->28340223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.400845:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28336128->28340223] (req 28336128->28340223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760797.431277:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28336128 -> 28340224 00010000:00010000:2.0:1789760797.431324:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28336128->28340223] (req 28336128->28340223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.431337:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484f6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28336128->28340223] (req 28336128->28340223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.431346:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28336128->28340223] (req 28336128->28340223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.431353:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760797.431357:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28336128->28340223] (req 28336128->28340223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.431366:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484f6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28336128->28340223] (req 28336128->28340223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.431482:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484f6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28336128->28340223] (req 28336128->28340223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.431492:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x49367483545484f6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28336128->28340223] (req 28336128->28340223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.431589:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2359:x1876699495970048:12345-192.168.203.9@tcp:4:dd.0 Request processed in 30839us (31094us total) trans 4294974235 rc 0/0 00000100:00100000:2.0:1789760797.431600:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6947 00000100:00100000:2.0:1789760797.432057:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781683920471 time 781721451617 state 0x2 ref 0 00000001:00080000:0.0:1789760797.434100:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974235 is committed 00000100:00100000:1.0:1789760797.436344:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760797.436517:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495970176 00000100:00100000:3.0:1789760797.436528:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495970176 00000100:00100000:3.0:1789760797.436583:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781683920471 time 781721451617 state 0x0 ref 1 00000100:00100000:3.0:1789760797.436599:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781721712033 time 781721451617 state 0x1 ref 1 00000100:00100000:3.0:1789760797.436695:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781721712033 time 781757410004 state 0x0 ref 1 00000100:00100000:3.0:1789760797.436707:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495970176:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760797.436756:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28340224, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:3.0:1789760797.436765:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484fd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.436781:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760797.436785:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.436793:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28340224->28344319] (req 28340224->28344319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.436807:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28340224->28344319] (req 28340224->28344319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.483807:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974236 is committed 00002000:00010000:1.0:1789760797.484014:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28340224 -> 28344320 00010000:00010000:1.0:1789760797.484030:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28340224->28344319] (req 28340224->28344319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.484044:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484fd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28340224->28344319] (req 28340224->28344319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.484053:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28340224->28344319] (req 28340224->28344319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.484066:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ce00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760797.484071:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28340224->28344319] (req 28340224->28344319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.484080:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484fd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28340224->28344319] (req 28340224->28344319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.484101:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484fd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28340224->28344319] (req 28340224->28344319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.484109:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f79b36f5/0x49367483545484fd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28340224->28344319] (req 28340224->28344319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.484203:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04930e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495970176:12345-192.168.203.9@tcp:4:dd.0 Request processed in 47492us (47864us total) trans 4294974236 rc 0/0 00000100:00100000:1.0:1789760797.484858:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6948 00000100:00100000:1.0:1789760797.484869:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781721712033 time 781757410004 state 0x2 ref 0 00000100:00100000:1.0:1789760797.489700:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760797.489803:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495970304 00000100:00100000:3.0:1789760797.489813:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495970304 00000100:00100000:3.0:1789760797.489866:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781721712033 time 781757410004 state 0x0 ref 1 00000100:00100000:3.0:1789760797.489881:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781757670420 time 781757410004 state 0x1 ref 1 00000100:00100000:3.0:1789760797.489914:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781757670420 time 781810631191 state 0x0 ref 1 00000100:00100000:3.0:1789760797.489926:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04930380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495970304:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760797.489978:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28344320, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:3.0:1789760797.489989:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548504 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.490006:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760797.490010:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.490017:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548504 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28344320->28348415] (req 28344320->28348415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.490032:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548504 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28344320->28348415] (req 28344320->28348415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.534355:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760797.534692:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974237 is committed 00002000:00010000:1.0:1789760797.534868:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28344320 -> 28348416 00010000:00010000:1.0:1789760797.534885:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548504 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28344320->28348415] (req 28344320->28348415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.534898:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548504 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28344320->28348415] (req 28344320->28348415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.534912:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548504 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28344320->28348415] (req 28344320->28348415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.534921:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760797.534926:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548504 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28344320->28348415] (req 28344320->28348415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.534936:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548504 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28344320->28348415] (req 28344320->28348415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.535068:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548504 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28344320->28348415] (req 28344320->28348415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.535078:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c78a56c/0x4936748354548504 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28344320->28348415] (req 28344320->28348415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.535215:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04930380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495970304:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45282us (45516us total) trans 4294974237 rc 0/0 00000100:00100000:1.0:1789760797.535225:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6949 00000100:00100000:1.0:1789760797.535235:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781757670420 time 781810631191 state 0x2 ref 0 00000100:00100000:0.0:1789760797.539910:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760797.540080:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495970432 00000100:00100000:2.0:1789760797.540091:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495970432 00000100:00100000:2.0:1789760797.540148:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781757670420 time 781810631191 state 0x0 ref 1 00000100:00100000:2.0:1789760797.540164:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781810891607 time 781810631191 state 0x1 ref 1 00000100:00100000:2.0:1789760797.540201:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781810891607 time 781860917267 state 0x0 ref 1 00000100:00100000:2.0:1789760797.540214:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495970432:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760797.540791:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28348416, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:2.0:1789760797.540803:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454850b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.540821:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=781s v=(0 0) 00010000:00010000:2.0:1789760797.540826:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.540833:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454850b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28348416->28352511] (req 28348416->28352511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.540847:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454850b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28348416->28352511] (req 28348416->28352511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.575597:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974238 is committed 00002000:00010000:3.0:1789760797.575818:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28348416 -> 28352512 00010000:00010000:3.0:1789760797.575834:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454850b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28348416->28352511] (req 28348416->28352511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.575912:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454850b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28348416->28352511] (req 28348416->28352511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.575927:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454850b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28348416->28352511] (req 28348416->28352511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.575935:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760797.575940:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454850b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28348416->28352511] (req 28348416->28352511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.575949:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454850b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28348416->28352511] (req 28348416->28352511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.575969:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454850b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28348416->28352511] (req 28348416->28352511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.575976:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454850b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28348416->28352511] (req 28348416->28352511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.576103:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c6a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495970432:12345-192.168.203.9@tcp:4:dd.0 Request processed in 35886us (36197us total) trans 4294974238 rc 0/0 00000100:00100000:3.0:1789760797.576113:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6950 00000100:00100000:3.0:1789760797.576122:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781810891607 time 781860917267 state 0x2 ref 0 00000100:00100000:0.0:1789760797.585553:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760797.585718:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495970560 00000100:00100000:1.0:1789760797.585728:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495970560 00000100:00100000:1.0:1789760797.585781:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781810891607 time 781860917267 state 0x0 ref 1 00000100:00100000:1.0:1789760797.585808:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781861177683 time 781860917267 state 0x1 ref 1 00000100:00100000:1.0:1789760797.585836:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781861177683 time 781906552778 state 0x0 ref 1 00000100:00100000:1.0:1789760797.585855:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495970560:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760797.585906:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28352512, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:1.0:1789760797.585915:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354548512 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.585930:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=781s v=(0 0) 00010000:00010000:1.0:1789760797.585935:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.585941:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354548512 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28352512->28356607] (req 28352512->28356607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.585954:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354548512 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28352512->28356607] (req 28352512->28356607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.652222:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974239 is committed 00002000:00010000:3.0:1789760797.652335:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28352512 -> 28356608 00010000:00010000:3.0:1789760797.652351:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354548512 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28352512->28356607] (req 28352512->28356607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.652370:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354548512 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28352512->28356607] (req 28352512->28356607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.652379:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354548512 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28352512->28356607] (req 28352512->28356607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.652386:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760797.652390:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354548512 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28352512->28356607] (req 28352512->28356607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.652400:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354548512 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28352512->28356607] (req 28352512->28356607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.652419:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354548512 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28352512->28356607] (req 28352512->28356607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.652426:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x4936748354548512 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28352512->28356607] (req 28352512->28356607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.652513:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495970560:12345-192.168.203.9@tcp:4:dd.0 Request processed in 66655us (66963us total) trans 4294974239 rc 0/0 00000100:00100000:3.0:1789760797.652523:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6951 00000100:00100000:3.0:1789760797.652533:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781861177683 time 781906552778 state 0x2 ref 0 00000100:00100000:1.0:1789760797.657831:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760797.657921:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495970688 00000100:00100000:3.0:1789760797.657931:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495970688 00000100:00100000:3.0:1789760797.657984:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781861177683 time 781906552778 state 0x0 ref 1 00000100:00100000:3.0:1789760797.657999:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781906813194 time 781906552778 state 0x1 ref 1 00000100:00100000:3.0:1789760797.658035:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781906813194 time 781978751692 state 0x0 ref 1 00000100:00100000:3.0:1789760797.658047:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04932d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495970688:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760797.658094:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28356608, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:3.0:1789760797.658104:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354548519 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.658122:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760797.658126:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.658132:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354548519 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28356608->28360703] (req 28356608->28360703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.658144:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354548519 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28356608->28360703] (req 28356608->28360703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.688891:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974240 is committed 00002000:00010000:1.0:1789760797.688999:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28356608 -> 28360704 00002000:00010000:1.0:1789760797.689003:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760796 -> 1789760797 00002000:00010000:1.0:1789760797.689005:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760796 -> 1789760797 00010000:00010000:1.0:1789760797.689019:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354548519 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28356608->28360703] (req 28356608->28360703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.689032:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354548519 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28356608->28360703] (req 28356608->28360703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.689040:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354548519 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28356608->28360703] (req 28356608->28360703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.689047:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081ca00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760797.689052:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354548519 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28356608->28360703] (req 28356608->28360703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.689060:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354548519 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28356608->28360703] (req 28356608->28360703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.689080:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354548519 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28356608->28360703] (req 28356608->28360703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.689087:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001525b470/0x4936748354548519 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28356608->28360703] (req 28356608->28360703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.689179:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04932d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495970688:12345-192.168.203.9@tcp:4:dd.0 Request processed in 31128us (31352us total) trans 4294974240 rc 0/0 00000100:00100000:1.0:1789760797.689189:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6952 00000100:00100000:1.0:1789760797.689199:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781906813194 time 781978751692 state 0x2 ref 0 00000100:00100000:0.0:1789760797.693580:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760797.693723:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495970816 00000100:00100000:3.0:1789760797.693734:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495970816 00000100:00100000:3.0:1789760797.693790:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781906813194 time 781978751692 state 0x0 ref 1 00000100:00100000:3.0:1789760797.693806:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781979012108 time 781978751692 state 0x1 ref 1 00000100:00100000:3.0:1789760797.693837:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781979012108 time 782014553483 state 0x0 ref 1 00000100:00100000:3.0:1789760797.693854:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495970816:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760797.693903:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28360704, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:3.0:1789760797.693911:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548520 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.693927:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=782s v=(0 0) 00010000:00010000:3.0:1789760797.693932:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.693940:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548520 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28360704->28364799] (req 28360704->28364799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.694072:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548520 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28360704->28364799] (req 28360704->28364799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760797.731484:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28360704 -> 28364800 00010000:00010000:2.0:1789760797.731505:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548520 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28360704->28364799] (req 28360704->28364799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.731520:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548520 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28360704->28364799] (req 28360704->28364799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.731528:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548520 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28360704->28364799] (req 28360704->28364799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.731535:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760797.731541:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548520 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28360704->28364799] (req 28360704->28364799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.731550:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548520 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28360704->28364799] (req 28360704->28364799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.731570:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548520 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28360704->28364799] (req 28360704->28364799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.731578:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001c7fee77/0x4936748354548520 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28360704->28364799] (req 28360704->28364799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.731688:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495970816:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37832us (38114us total) trans 4294974241 rc 0/0 00000100:00100000:2.0:1789760797.731699:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6953 00000100:00100000:2.0:1789760797.731710:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781979012108 time 782014553483 state 0x2 ref 0 00000001:00080000:1.0:1789760797.734107:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974241 is committed 00000100:00100000:0.0:1789760797.736016:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760797.736095:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495970944 00000100:00100000:3.0:1789760797.736106:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495970944 00000100:00100000:3.0:1789760797.736162:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 781979012108 time 782014553483 state 0x0 ref 1 00000100:00100000:3.0:1789760797.736178:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782014813899 time 782014553483 state 0x1 ref 1 00000100:00100000:3.0:1789760797.736239:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782014813899 time 782056955684 state 0x0 ref 1 00000100:00100000:3.0:1789760797.736252:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92900c7cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495970944:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:3.0:1789760797.736320:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28364800, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:3.0:1789760797.736328:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548527 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.736349:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=782s v=(0 0) 00010000:00010000:3.0:1789760797.736354:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.736361:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548527 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28364800->28368895] (req 28364800->28368895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.736373:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548527 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28364800->28368895] (req 28364800->28368895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.770650:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974242 is committed 00002000:00010000:1.0:1789760797.770809:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28364800 -> 28368896 00010000:00010000:1.0:1789760797.770825:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548527 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28364800->28368895] (req 28364800->28368895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.770838:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548527 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28364800->28368895] (req 28364800->28368895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.770853:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548527 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28364800->28368895] (req 28364800->28368895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.770861:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92904081d000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760797.770866:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548527 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28364800->28368895] (req 28364800->28368895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.770875:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548527 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28364800->28368895] (req 28364800->28368895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.770894:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548527 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28364800->28368895] (req 28364800->28368895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.770901:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319ecbd7/0x4936748354548527 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28364800->28368895] (req 28364800->28368895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.771070:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92900c7cf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495970944:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34815us (35058us total) trans 4294974242 rc 0/0 00000100:00100000:1.0:1789760797.771081:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6954 00000100:00100000:1.0:1789760797.771091:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782014813899 time 782056955684 state 0x2 ref 0 00000100:00100000:0.0:1789760797.774795:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760797.774878:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495971072 00000100:00100000:1.0:1789760797.774888:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495971072 00000100:00100000:1.0:1789760797.774940:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782014813899 time 782056955684 state 0x0 ref 1 00000100:00100000:1.0:1789760797.774964:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782057216100 time 782056955684 state 0x1 ref 1 00000100:00100000:1.0:1789760797.774995:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782057216100 time 782095711423 state 0x0 ref 1 00000100:00100000:1.0:1789760797.775008:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495971072:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760797.775055:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28368896, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:1.0:1789760797.775065:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454852e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.775080:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=782s v=(0 0) 00010000:00010000:1.0:1789760797.775084:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.775091:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454852e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28368896->28372991] (req 28368896->28372991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.775104:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454852e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28368896->28372991] (req 28368896->28372991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760797.802215:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28368896 -> 28372992 00010000:00010000:1.0:1789760797.802237:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454852e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28368896->28372991] (req 28368896->28372991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.802251:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454852e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28368896->28372991] (req 28368896->28372991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.802260:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454852e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28368896->28372991] (req 28368896->28372991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.802266:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760797.802273:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454852e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28368896->28372991] (req 28368896->28372991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.802486:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454852e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28368896->28372991] (req 28368896->28372991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.802509:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454852e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28368896->28372991] (req 28368896->28372991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.802517:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454852e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28368896->28372991] (req 28368896->28372991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.802752:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2356:x1876699495971072:12345-192.168.203.9@tcp:4:dd.0 Request processed in 27740us (27959us total) trans 4294974243 rc 0/0 00000100:00100000:1.0:1789760797.802763:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6955 00000100:00100000:1.0:1789760797.802772:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782057216100 time 782095711423 state 0x2 ref 0 00000001:00080000:1.0:1789760797.802816:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974243 is committed 00000100:00100000:0.0:1789760797.811585:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760797.811849:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495971200 00000100:00100000:2.0:1789760797.811859:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495971200 00000100:00100000:2.0:1789760797.811913:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782057216100 time 782095711423 state 0x0 ref 1 00000100:00100000:2.0:1789760797.811928:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782095971839 time 782095711423 state 0x1 ref 1 00000100:00100000:2.0:1789760797.811969:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782095971839 time 782132685930 state 0x0 ref 1 00000100:00100000:2.0:1789760797.811981:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04575500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495971200:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760797.812027:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28372992, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:2.0:1789760797.812037:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354548535 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.812053:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=782s v=(0 0) 00010000:00010000:2.0:1789760797.812058:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.812063:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354548535 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28372992->28377087] (req 28372992->28377087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.812077:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354548535 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28372992->28377087] (req 28372992->28377087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.853396:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974244 is committed 00002000:00010000:1.0:1789760797.853659:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28372992 -> 28377088 00010000:00010000:1.0:1789760797.853678:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354548535 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28372992->28377087] (req 28372992->28377087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.853692:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354548535 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28372992->28377087] (req 28372992->28377087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.853703:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354548535 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28372992->28377087] (req 28372992->28377087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.853717:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760797.853723:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354548535 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28372992->28377087] (req 28372992->28377087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.853734:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354548535 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28372992->28377087] (req 28372992->28377087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.853754:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354548535 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28372992->28377087] (req 28372992->28377087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.853763:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354548535 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28372992->28377087] (req 28372992->28377087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.853867:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04575500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495971200:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41882us (42285us total) trans 4294974244 rc 0/0 00000100:00100000:1.0:1789760797.853879:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6956 00000100:00100000:1.0:1789760797.853891:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782095971839 time 782132685930 state 0x2 ref 0 00000100:00100000:3.0:1789760797.870576:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760797.870989:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495971328 00000100:00100000:1.0:1789760797.871000:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495971328 00000100:00100000:1.0:1789760797.871065:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782095971839 time 782132685930 state 0x0 ref 1 00000100:00100000:1.0:1789760797.871079:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782132946346 time 782132685930 state 0x1 ref 1 00000100:00100000:1.0:1789760797.871112:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782132946346 time 782191829470 state 0x0 ref 1 00000100:00100000:1.0:1789760797.871126:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495971328:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760797.871170:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28377088, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:1.0:1789760797.871178:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454853c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.871194:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=782s v=(0 0) 00010000:00010000:1.0:1789760797.871198:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.871204:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454853c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28377088->28381183] (req 28377088->28381183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.871219:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454853c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28377088->28381183] (req 28377088->28381183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.930464:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974245 is committed 00002000:00010000:0.0:1789760797.933883:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28377088 -> 28381184 00010000:00010000:0.0:1789760797.933903:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454853c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28377088->28381183] (req 28377088->28381183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.933917:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454853c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28377088->28381183] (req 28377088->28381183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.933928:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454853c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28377088->28381183] (req 28377088->28381183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.933936:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7000 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760797.933941:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454853c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28377088->28381183] (req 28377088->28381183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.933950:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454853c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28377088->28381183] (req 28377088->28381183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.933970:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454853c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28377088->28381183] (req 28377088->28381183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.933979:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000092be3b0/0x493674835454853c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28377088->28381183] (req 28377088->28381183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.934084:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495971328:12345-192.168.203.9@tcp:4:dd.0 Request processed in 62954us (63512us total) trans 4294974245 rc 0/0 00000100:00100000:0.0:1789760797.934095:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6957 00000100:00100000:0.0:1789760797.934105:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782132946346 time 782191829470 state 0x2 ref 0 00000100:00100000:0.0:1789760797.940512:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760797.940666:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495971456 00000100:00100000:1.0:1789760797.940677:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495971456 00000100:00100000:1.0:1789760797.940732:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782132946346 time 782191829470 state 0x0 ref 1 00000100:00100000:1.0:1789760797.940748:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782192089886 time 782191829470 state 0x1 ref 1 00000100:00100000:1.0:1789760797.940779:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782192089886 time 782261496494 state 0x0 ref 1 00000100:00100000:1.0:1789760797.940792:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04577b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495971456:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760797.940840:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28381184, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:1.0:1789760797.940849:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548543 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.940865:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=782s v=(0 0) 00010000:00010000:1.0:1789760797.940870:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.940877:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548543 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28381184->28385279] (req 28381184->28385279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.940890:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548543 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28381184->28385279] (req 28381184->28385279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760798.004210:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28381184 -> 28385280 00010000:00010000:0.0:1789760798.004230:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548543 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28381184->28385279] (req 28381184->28385279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.004248:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548543 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28381184->28385279] (req 28381184->28385279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.004258:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548543 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28381184->28385279] (req 28381184->28385279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.004266:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760798.004271:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548543 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28381184->28385279] (req 28381184->28385279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.004309:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548543 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28381184->28385279] (req 28381184->28385279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.004329:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548543 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28381184->28385279] (req 28381184->28385279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.004336:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ffb10fa/0x4936748354548543 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28381184->28385279] (req 28381184->28385279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.004421:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04577b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495971456:12345-192.168.203.9@tcp:4:dd.0 Request processed in 63626us (63913us total) trans 4294974246 rc 0/0 00000100:00100000:0.0:1789760798.004432:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6958 00000100:00100000:0.0:1789760798.004443:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782192089886 time 782261496494 state 0x2 ref 0 00000001:00080000:0.0:1789760798.004494:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974246 is committed 00000100:00100000:1.0:1789760798.012128:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760798.012229:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495971584 00000100:00100000:0.0:1789760798.012240:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495971584 00000100:00100000:0.0:1789760798.012325:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782192089886 time 782261496494 state 0x0 ref 1 00000100:00100000:0.0:1789760798.012342:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782261756910 time 782261496494 state 0x1 ref 1 00000100:00100000:0.0:1789760798.012375:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782261756910 time 782333091829 state 0x0 ref 1 00000100:00100000:0.0:1789760798.012389:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04933100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495971584:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760798.012435:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28385280, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:0.0:1789760798.012444:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454854a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.012460:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=782s v=(0 0) 00010000:00010000:0.0:1789760798.012464:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.012470:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454854a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28385280->28389375] (req 28385280->28389375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.012484:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454854a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28385280->28389375] (req 28385280->28389375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.049499:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974247 is committed 00002000:00010000:1.0:1789760798.049679:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28385280 -> 28389376 00010000:00010000:1.0:1789760798.049694:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454854a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28385280->28389375] (req 28385280->28389375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.049707:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454854a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28385280->28389375] (req 28385280->28389375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.049716:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454854a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28385280->28389375] (req 28385280->28389375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.049724:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760798.049728:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454854a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28385280->28389375] (req 28385280->28389375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.049737:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454854a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28385280->28389375] (req 28385280->28389375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.049761:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454854a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28385280->28389375] (req 28385280->28389375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.049769:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073e6239e/0x493674835454854a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28385280->28389375] (req 28385280->28389375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.049860:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04933100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495971584:12345-192.168.203.9@tcp:4:dd.0 Request processed in 37468us (37735us total) trans 4294974247 rc 0/0 00000100:00100000:1.0:1789760798.049870:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6959 00000100:00100000:1.0:1789760798.049884:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782261756910 time 782333091829 state 0x2 ref 0 00000100:00100000:3.0:1789760798.055027:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760798.058074:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495971712 00000100:00100000:0.0:1789760798.058088:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495971712 00000100:00100000:0.0:1789760798.058148:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782261756910 time 782333091829 state 0x0 ref 1 00000100:00100000:0.0:1789760798.058162:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782333352245 time 782333091829 state 0x1 ref 1 00000100:00100000:0.0:1789760798.058196:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782333352245 time 782378913183 state 0x0 ref 1 00000100:00100000:0.0:1789760798.058209:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f043bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495971712:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760798.058252:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28389376, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:0.0:1789760798.058261:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548551 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.058275:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=782s v=(0 0) 00010000:00010000:0.0:1789760798.058295:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.058302:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548551 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28389376->28393471] (req 28389376->28393471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.058315:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548551 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28389376->28393471] (req 28389376->28393471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.105966:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974248 is committed 00002000:00010000:1.0:1789760798.106152:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28389376 -> 28393472 00010000:00010000:1.0:1789760798.106168:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548551 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28389376->28393471] (req 28389376->28393471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.106182:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548551 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28389376->28393471] (req 28389376->28393471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.106192:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548551 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28389376->28393471] (req 28389376->28393471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.106200:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94b800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760798.106204:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548551 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28389376->28393471] (req 28389376->28393471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.106219:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548551 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28389376->28393471] (req 28389376->28393471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.106239:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548551 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28389376->28393471] (req 28389376->28393471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.106248:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083bc5ef9/0x4936748354548551 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28389376->28393471] (req 28389376->28393471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.106403:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f043bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495971712:12345-192.168.203.9@tcp:4:dd.0 Request processed in 48191us (51380us total) trans 4294974248 rc 0/0 00000100:00100000:1.0:1789760798.106413:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6960 00000100:00100000:1.0:1789760798.106423:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782333352245 time 782378913183 state 0x2 ref 0 00000100:00100000:0.0:1789760798.109139:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760798.109349:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495971840 00000100:00100000:1.0:1789760798.109359:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495971840 00000100:00100000:1.0:1789760798.109411:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782333352245 time 782378913183 state 0x0 ref 1 00000100:00100000:1.0:1789760798.109426:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782379173599 time 782378913183 state 0x1 ref 1 00000100:00100000:1.0:1789760798.109459:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782379173599 time 782430175862 state 0x0 ref 1 00000100:00100000:1.0:1789760798.109472:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04929500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495971840:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760798.109519:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28393472, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:1.0:1789760798.109528:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354548558 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.109545:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=782s v=(0 0) 00010000:00010000:1.0:1789760798.109549:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.109556:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354548558 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28393472->28397567] (req 28393472->28397567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.109569:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354548558 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28393472->28397567] (req 28393472->28397567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.145441:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974249 is committed 00002000:00010000:1.0:1789760798.145543:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28393472 -> 28397568 00010000:00010000:1.0:1789760798.145559:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354548558 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28393472->28397567] (req 28393472->28397567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.145572:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354548558 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28393472->28397567] (req 28393472->28397567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.145581:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354548558 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28393472->28397567] (req 28393472->28397567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.145588:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760798.145592:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354548558 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28393472->28397567] (req 28393472->28397567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.145671:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354548558 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28393472->28397567] (req 28393472->28397567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.145692:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354548558 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28393472->28397567] (req 28393472->28397567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.145702:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045557e70/0x4936748354548558 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28393472->28397567] (req 28393472->28397567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.145796:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04929500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495971840:12345-192.168.203.9@tcp:4:dd.0 Request processed in 36321us (36661us total) trans 4294974249 rc 0/0 00000100:00100000:1.0:1789760798.145808:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6961 00000100:00100000:1.0:1789760798.145818:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782379173599 time 782430175862 state 0x2 ref 0 00000100:00100000:2.0:1789760798.149844:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760798.149922:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495971968 00000100:00100000:1.0:1789760798.149931:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495971968 00000100:00100000:1.0:1789760798.149993:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782379173599 time 782430175862 state 0x0 ref 1 00000100:00100000:1.0:1789760798.150009:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782430436278 time 782430175862 state 0x1 ref 1 00000100:00100000:1.0:1789760798.150037:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782430436278 time 782470754548 state 0x0 ref 1 00000100:00100000:1.0:1789760798.150050:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f0276ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495971968:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760798.150097:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28397568, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:1.0:1789760798.150107:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x493674835454855f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.150124:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=782s v=(0 0) 00010000:00010000:1.0:1789760798.150128:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.150134:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x493674835454855f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28397568->28401663] (req 28397568->28401663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.150152:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x493674835454855f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28397568->28401663] (req 28397568->28401663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760798.182812:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28397568 -> 28401664 00010000:00010000:2.0:1789760798.182835:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x493674835454855f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28397568->28401663] (req 28397568->28401663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.182849:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x493674835454855f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28397568->28401663] (req 28397568->28401663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.182858:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x493674835454855f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28397568->28401663] (req 28397568->28401663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.182866:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6a00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760798.182870:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x493674835454855f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28397568->28401663] (req 28397568->28401663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.182880:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x493674835454855f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28397568->28401663] (req 28397568->28401663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.182900:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x493674835454855f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28397568->28401663] (req 28397568->28401663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.182908:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee6f90eb/0x493674835454855f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28397568->28401663] (req 28397568->28401663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.183019:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f0276ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495971968:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32965us (33179us total) trans 4294974250 rc 0/0 00000100:00100000:2.0:1789760798.183031:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6962 00000100:00100000:2.0:1789760798.183041:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782430436278 time 782470754548 state 0x2 ref 0 00000001:00080000:3.0:1789760798.184168:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974250 is committed 00000100:00100000:3.0:1789760798.189676:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760798.189778:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495972096 00000100:00100000:1.0:1789760798.189787:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495972096 00000100:00100000:1.0:1789760798.189839:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782430436278 time 782470754548 state 0x0 ref 1 00000100:00100000:1.0:1789760798.189853:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782471014964 time 782470754548 state 0x1 ref 1 00000100:00100000:1.0:1789760798.189884:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782471014964 time 782510601211 state 0x0 ref 1 00000100:00100000:1.0:1789760798.189896:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01ac5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495972096:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760798.189940:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28401664, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:1.0:1789760798.189949:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354548566 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.189965:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=782s v=(0 0) 00010000:00010000:1.0:1789760798.189969:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.189975:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354548566 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28401664->28405759] (req 28401664->28405759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.189987:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354548566 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28401664->28405759] (req 28401664->28405759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760798.223411:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28401664 -> 28405760 00010000:00010000:1.0:1789760798.223434:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354548566 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28401664->28405759] (req 28401664->28405759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.223448:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354548566 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28401664->28405759] (req 28401664->28405759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.223458:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354548566 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28401664->28405759] (req 28401664->28405759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.223466:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6c00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760798.223472:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354548566 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28401664->28405759] (req 28401664->28405759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.223481:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354548566 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28401664->28405759] (req 28401664->28405759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.223503:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354548566 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28401664->28405759] (req 28401664->28405759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.223512:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d870953/0x4936748354548566 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28401664->28405759] (req 28401664->28405759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760798.223791:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974251 is committed 00000100:00100000:1.0:1789760798.223918:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01ac5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2356:x1876699495972096:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34018us (34244us total) trans 4294974251 rc 0/0 00000100:00100000:1.0:1789760798.223929:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6963 00000100:00100000:1.0:1789760798.223939:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782471014964 time 782510601211 state 0x2 ref 0 00000100:00100000:1.0:1789760798.231626:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760798.231813:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495972224 00000100:00100000:0.0:1789760798.231823:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495972224 00000100:00100000:0.0:1789760798.231878:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782471014964 time 782510601211 state 0x0 ref 1 00000100:00100000:0.0:1789760798.231893:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782510861627 time 782510601211 state 0x1 ref 1 00000100:00100000:0.0:1789760798.231968:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782510861627 time 782552683485 state 0x0 ref 1 00000100:00100000:0.0:1789760798.231984:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04933480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495972224:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760798.232033:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28405760, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:0.0:1789760798.232042:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454856d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.232058:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=782s v=(0 0) 00010000:00010000:0.0:1789760798.232063:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.232071:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454856d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28405760->28409855] (req 28405760->28409855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.232085:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454856d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28405760->28409855] (req 28405760->28409855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760798.276567:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974252 is committed 00002000:00010000:1.0:1789760798.276861:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28405760 -> 28409856 00010000:00010000:1.0:1789760798.276878:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454856d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28405760->28409855] (req 28405760->28409855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.276891:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454856d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28405760->28409855] (req 28405760->28409855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.276899:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454856d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28405760->28409855] (req 28405760->28409855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.276906:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94a600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760798.276911:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454856d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28405760->28409855] (req 28405760->28409855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.276919:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454856d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28405760->28409855] (req 28405760->28409855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.276939:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454856d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28405760->28409855] (req 28405760->28409855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.276947:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5eeea32/0x493674835454856d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28405760->28409855] (req 28405760->28409855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.277046:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04933480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495972224:12345-192.168.203.9@tcp:4:dd.0 Request processed in 45058us (45435us total) trans 4294974252 rc 0/0 00000100:00100000:1.0:1789760798.277061:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6964 00000100:00100000:1.0:1789760798.277072:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782510861627 time 782552683485 state 0x2 ref 0 00000100:00100000:0.0:1789760798.283023:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760798.283145:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495972352 00000100:00100000:2.0:1789760798.283156:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495972352 00000100:00100000:2.0:1789760798.283218:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782510861627 time 782552683485 state 0x0 ref 1 00000100:00100000:2.0:1789760798.283234:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782552943901 time 782552683485 state 0x1 ref 1 00000100:00100000:2.0:1789760798.283339:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782552943901 time 782604054909 state 0x0 ref 1 00000100:00100000:2.0:1789760798.283354:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f03095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495972352:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760798.283404:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28409856, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:2.0:1789760798.283414:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548574 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.283431:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=782s v=(0 0) 00010000:00010000:2.0:1789760798.283435:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.283441:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548574 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28409856->28413951] (req 28409856->28413951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.283454:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548574 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28409856->28413951] (req 28409856->28413951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760798.323401:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974253 is committed 00002000:00010000:0.0:1789760798.323665:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28409856 -> 28413952 00010000:00010000:0.0:1789760798.323680:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548574 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28409856->28413951] (req 28409856->28413951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.323692:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548574 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28409856->28413951] (req 28409856->28413951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.323700:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548574 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28409856->28413951] (req 28409856->28413951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.323708:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fee00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760798.323713:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548574 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28409856->28413951] (req 28409856->28413951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.323723:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548574 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28409856->28413951] (req 28409856->28413951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.323743:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548574 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28409856->28413951] (req 28409856->28413951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.323750:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003589f2e1/0x4936748354548574 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28409856->28413951] (req 28409856->28413951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.323838:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f03095c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495972352:12345-192.168.203.9@tcp:4:dd.0 Request processed in 40480us (40819us total) trans 4294974253 rc 0/0 00000100:00100000:0.0:1789760798.323848:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6965 00000100:00100000:0.0:1789760798.323858:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782552943901 time 782604054909 state 0x2 ref 0 00000100:00100000:3.0:1789760798.329196:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760798.329585:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495972480 00000100:00100000:1.0:1789760798.329610:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495972480 00000100:00100000:1.0:1789760798.329674:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782552943901 time 782604054909 state 0x0 ref 1 00000100:00100000:1.0:1789760798.330207:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782604315325 time 782604054909 state 0x1 ref 1 00000100:00100000:1.0:1789760798.330244:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782604315325 time 782650960066 state 0x0 ref 1 00000100:00100000:1.0:1789760798.330258:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f01374a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495972480:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760798.330346:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28413952, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:1.0:1789760798.330356:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454857b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.330372:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=782s v=(0 0) 00010000:00010000:1.0:1789760798.330377:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.330383:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454857b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28413952->28418047] (req 28413952->28418047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.330399:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454857b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28413952->28418047] (req 28413952->28418047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.363173:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974254 is committed 00002000:00010000:2.0:1789760798.363336:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28413952 -> 28418048 00010000:00010000:2.0:1789760798.363352:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454857b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28413952->28418047] (req 28413952->28418047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.363371:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454857b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28413952->28418047] (req 28413952->28418047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.363380:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454857b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28413952->28418047] (req 28413952->28418047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.363388:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf6800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760798.363393:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454857b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28413952->28418047] (req 28413952->28418047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.363404:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454857b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28413952->28418047] (req 28413952->28418047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.363424:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454857b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28413952->28418047] (req 28413952->28418047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.363432:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b1ebd15/0x493674835454857b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28413952->28418047] (req 28413952->28418047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.363521:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f01374a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495972480:12345-192.168.203.9@tcp:4:dd.0 Request processed in 33263us (34330us total) trans 4294974254 rc 0/0 00000100:00100000:2.0:1789760798.363533:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6966 00000100:00100000:2.0:1789760798.363542:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782604315325 time 782650960066 state 0x2 ref 0 00000100:00100000:1.0:1789760798.372230:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:0.0:1789760798.372377:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495972608 00000100:00100000:0.0:1789760798.372393:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495972608 00000100:00100000:0.0:1789760798.372458:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782604315325 time 782650960066 state 0x0 ref 1 00000100:00100000:0.0:1789760798.372475:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782651220482 time 782650960066 state 0x1 ref 1 00000100:00100000:0.0:1789760798.372503:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782651220482 time 782693220547 state 0x0 ref 1 00000100:00100000:0.0:1789760798.372516:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04932680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495972608:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:0.0:1789760798.372576:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28418048, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:0.0:1789760798.372586:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354548582 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.372603:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=782s v=(0 0) 00010000:00010000:0.0:1789760798.372626:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.372633:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354548582 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28418048->28422143] (req 28418048->28422143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.372647:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354548582 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28418048->28422143] (req 28418048->28422143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.421304:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974255 is committed 00002000:00010000:2.0:1789760798.421411:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28418048 -> 28422144 00010000:00010000:2.0:1789760798.421426:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354548582 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28418048->28422143] (req 28418048->28422143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.421448:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354548582 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28418048->28422143] (req 28418048->28422143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.421456:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354548582 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28418048->28422143] (req 28418048->28422143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.421462:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff92903f94bc00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760798.421467:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354548582 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28418048->28422143] (req 28418048->28422143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.421475:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354548582 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28418048->28422143] (req 28418048->28422143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.421494:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354548582 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28418048->28422143] (req 28418048->28422143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.421501:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023c8d681/0x4936748354548582 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28418048->28422143] (req 28418048->28422143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.421604:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04932680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495972608:12345-192.168.203.9@tcp:4:dd.0 Request processed in 49077us (49377us total) trans 4294974255 rc 0/0 00000100:00100000:2.0:1789760798.421623:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6967 00000100:00100000:2.0:1789760798.421635:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782651220482 time 782693220547 state 0x2 ref 0 00000100:00100000:0.0:1789760798.426904:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760798.426967:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495972736 00000100:00100000:2.0:1789760798.426980:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495972736 00000100:00100000:2.0:1789760798.427048:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782651220482 time 782693220547 state 0x0 ref 1 00000100:00100000:2.0:1789760798.427546:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782693480963 time 782693220547 state 0x1 ref 1 00000100:00100000:2.0:1789760798.427580:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782693480963 time 782748296336 state 0x0 ref 1 00000100:00100000:2.0:1789760798.427598:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495972736:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760798.427900:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28422144, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:2.0:1789760798.427910:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354548589 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.427925:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=782s v=(0 0) 00010000:00010000:2.0:1789760798.427930:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.427936:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354548589 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28422144->28426239] (req 28422144->28426239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.427948:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354548589 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28422144->28426239] (req 28422144->28426239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.468323:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974256 is committed 00002000:00010000:0.0:1789760798.468460:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28422144 -> 28426240 00010000:00010000:0.0:1789760798.468476:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354548589 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28422144->28426239] (req 28422144->28426239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.468489:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354548589 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28422144->28426239] (req 28422144->28426239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.468500:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354548589 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28422144->28426239] (req 28422144->28426239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.468507:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff400 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760798.468513:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354548589 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28422144->28426239] (req 28422144->28426239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.468521:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354548589 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28422144->28426239] (req 28422144->28426239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.468544:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354548589 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28422144->28426239] (req 28422144->28426239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.468552:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1ea9554/0x4936748354548589 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28422144->28426239] (req 28422144->28426239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.468748:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f925c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495972736:12345-192.168.203.9@tcp:4:dd.0 Request processed in 41149us (41854us total) trans 4294974256 rc 0/0 00000100:00100000:0.0:1789760798.468759:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6968 00000100:00100000:0.0:1789760798.468769:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782693480963 time 782748296336 state 0x2 ref 0 00000100:00100000:1.0:1789760798.471959:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760798.472045:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495972864 00000100:00100000:2.0:1789760798.472055:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495972864 00000100:00100000:2.0:1789760798.472116:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782693480963 time 782748296336 state 0x0 ref 1 00000100:00100000:2.0:1789760798.472130:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782748556752 time 782748296336 state 0x1 ref 1 00000100:00100000:2.0:1789760798.472163:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782748556752 time 782792880101 state 0x0 ref 1 00000100:00100000:2.0:1789760798.472175:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04930a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495972864:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760798.472219:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28426240, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:2.0:1789760798.472228:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354548590 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.472244:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=782s v=(0 0) 00010000:00010000:2.0:1789760798.472248:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.472254:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354548590 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28426240->28430335] (req 28426240->28430335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.472267:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354548590 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28426240->28430335] (req 28426240->28430335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760798.506697:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974257 is committed 00002000:00010000:1.0:1789760798.506766:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28426240 -> 28430336 00010000:00010000:1.0:1789760798.506780:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354548590 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28426240->28430335] (req 28426240->28430335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.506793:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354548590 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28426240->28430335] (req 28426240->28430335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.506803:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354548590 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28426240->28430335] (req 28426240->28430335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.506810:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ffe00 already unused, calling callback (ffffffffc1598350) 00010000:00010000:1.0:1789760798.506815:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354548590 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28426240->28430335] (req 28426240->28430335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.506825:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354548590 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28426240->28430335] (req 28426240->28430335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.506903:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354548590 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28426240->28430335] (req 28426240->28430335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.506911:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bb578e5/0x4936748354548590 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28426240->28430335] (req 28426240->28430335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.507028:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04930a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495972864:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34850us (35072us total) trans 4294974257 rc 0/0 00000100:00100000:1.0:1789760798.507041:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6969 00000100:00100000:1.0:1789760798.507050:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782748556752 time 782792880101 state 0x2 ref 0 00000100:00100000:1.0:1789760798.517909:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760798.518012:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495972992 00000100:00100000:2.0:1789760798.518023:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495972992 00000100:00100000:2.0:1789760798.518080:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782748556752 time 782792880101 state 0x0 ref 1 00000100:00100000:2.0:1789760798.518097:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782793140517 time 782792880101 state 0x1 ref 1 00000100:00100000:2.0:1789760798.518129:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782793140517 time 782838845461 state 0x0 ref 1 00000100:00100000:2.0:1789760798.518142:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f04931180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495972992:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760798.518191:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28430336, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:2.0:1789760798.518201:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354548597 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.518218:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=782s v=(0 0) 00010000:00010000:2.0:1789760798.518223:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.518228:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354548597 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28430336->28434431] (req 28430336->28434431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.518242:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354548597 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28430336->28434431] (req 28430336->28434431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.558415:0:8519:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760798.586349:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974258 is committed 00002000:00010000:2.0:1789760798.587718:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28430336 -> 28434432 00010000:00010000:2.0:1789760798.587738:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354548597 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28430336->28434431] (req 28430336->28434431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.587751:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354548597 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28430336->28434431] (req 28430336->28434431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.587760:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354548597 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28430336->28434431] (req 28430336->28434431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.587768:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760798.587772:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354548597 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28430336->28434431] (req 28430336->28434431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.587782:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354548597 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28430336->28434431] (req 28430336->28434431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.587808:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354548597 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28430336->28434431] (req 28430336->28434431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.587815:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x4936748354548597 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28430336->28434431] (req 28430336->28434431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.587974:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff928f04931180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495972992:12345-192.168.203.9@tcp:4:dd.0 Request processed in 69830us (70134us total) trans 4294974258 rc 0/0 00000100:00100000:2.0:1789760798.587985:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6970 00000100:00100000:2.0:1789760798.587995:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782793140517 time 782838845461 state 0x2 ref 0 00000100:00100000:0.0:1789760798.600886:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760798.601099:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495973120 00000100:00100000:2.0:1789760798.601109:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495973120 00000100:00100000:2.0:1789760798.601162:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782793140517 time 782838845461 state 0x0 ref 1 00000100:00100000:2.0:1789760798.601178:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782839105877 time 782838845461 state 0x1 ref 1 00000100:00100000:2.0:1789760798.601212:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782839105877 time 782921928119 state 0x0 ref 1 00000100:00100000:2.0:1789760798.601225:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495973120:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760798.601276:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28434432, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:2.0:1789760798.601422:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454859e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.601439:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=782s v=(0 0) 00010000:00010000:2.0:1789760798.601445:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.601451:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454859e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28434432->28438527] (req 28434432->28438527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.601698:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454859e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28434432->28438527] (req 28434432->28438527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.633311:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974259 is committed 00002000:00010000:3.0:1789760798.633397:0:6714:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28434432 -> 28438528 00010000:00010000:3.0:1789760798.633411:0:6714:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454859e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28434432->28438527] (req 28434432->28438527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.633425:0:6714:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454859e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28434432->28438527] (req 28434432->28438527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.633434:0:6714:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454859e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28434432->28438527] (req 28434432->28438527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.633442:0:6714:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062fe800 already unused, calling callback (ffffffffc1598350) 00010000:00010000:3.0:1789760798.633448:0:6714:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454859e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28434432->28438527] (req 28434432->28438527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.633456:0:6714:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454859e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28434432->28438527] (req 28434432->28438527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.633476:0:6714:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454859e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28434432->28438527] (req 28434432->28438527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.633483:0:6714:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a620e7/0x493674835454859e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28434432->28438527] (req 28434432->28438527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.633569:0:6714:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f924380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495973120:12345-192.168.203.9@tcp:4:dd.0 Request processed in 32340us (32688us total) trans 4294974259 rc 0/0 00000100:00100000:3.0:1789760798.633580:0:6714:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6971 00000100:00100000:3.0:1789760798.633591:0:6714:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782839105877 time 782921928119 state 0x2 ref 0 00000100:00100000:0.0:1789760798.647954:0:3310:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:1.0:1789760798.650471:0:13055:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495973248 00000100:00100000:1.0:1789760798.650497:0:13055:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495973248 00000100:00100000:1.0:1789760798.650562:0:13055:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782839105877 time 782921928119 state 0x0 ref 1 00000100:00100000:1.0:1789760798.650587:0:13055:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782922188535 time 782921928119 state 0x1 ref 1 00000100:00100000:1.0:1789760798.650635:0:13055:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782922188535 time 782971351447 state 0x0 ref 1 00000100:00100000:1.0:1789760798.650650:0:13055:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495973248:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:1.0:1789760798.650697:0:13055:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28438528, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:1.0:1789760798.650716:0:13055:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545485a5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.650732:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=782s v=(0 0) 00010000:00010000:1.0:1789760798.650736:0:13055:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.650743:0:13055:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545485a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28438528->28442623] (req 28438528->28442623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.650758:0:13055:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545485a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28438528->28442623] (req 28438528->28442623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.702490:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974260 is committed 00002000:00010000:0.0:1789760798.702661:0:13055:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28438528 -> 28442624 00002000:00010000:0.0:1789760798.702666:0:13055:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760797 -> 1789760798 00002000:00010000:0.0:1789760798.702669:0:13055:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760797 -> 1789760798 00010000:00010000:0.0:1789760798.702683:0:13055:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545485a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28438528->28442623] (req 28438528->28442623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.702696:0:13055:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545485a5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28438528->28442623] (req 28438528->28442623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.702706:0:13055:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545485a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28438528->28442623] (req 28438528->28442623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.702719:0:13055:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff929016bf7200 already unused, calling callback (ffffffffc1598350) 00010000:00010000:0.0:1789760798.702724:0:13055:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545485a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28438528->28442623] (req 28438528->28442623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.702733:0:13055:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545485a5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28438528->28442623] (req 28438528->28442623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.702756:0:13055:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545485a5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28438528->28442623] (req 28438528->28442623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.702763:0:13055:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5059f55/0x49367483545485a5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28438528->28442623] (req 28438528->28442623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13055 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.702866:0:13055:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff92903f926d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:17022965-1e4f-46fb-915c-61730096b8e2+5:2359:x1876699495973248:12345-192.168.203.9@tcp:4:dd.0 Request processed in 52215us (54917us total) trans 4294974260 rc 0/0 00000100:00100000:0.0:1789760798.702876:0:13055:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6972 00000100:00100000:0.0:1789760798.702886:0:13055:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782922188535 time 782971351447 state 0x2 ref 0 00000100:00100000:1.0:1789760798.706096:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:2.0:1789760798.706201:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495973376 00000100:00100000:2.0:1789760798.706211:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495973376 00000100:00100000:2.0:1789760798.706264:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782922188535 time 782971351447 state 0x0 ref 1 00000100:00100000:2.0:1789760798.706276:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782971611863 time 782971351447 state 0x1 ref 1 00000100:00100000:2.0:1789760798.706484:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782971611863 time 783027201676 state 0x0 ref 1 00000100:00100000:2.0:1789760798.706497:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff929018a80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+5:2357:x1876699495973376:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760798.706543:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28442624, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xe009 00010000:00010000:2.0:1789760798.706552:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545485ac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.706568:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=783s v=(0 0) 00010000:00010000:2.0:1789760798.706572:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.706576:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545485ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28442624->28446719] (req 28442624->28446719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.706588:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545485ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28442624->28446719] (req 28442624->28446719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760798.741010:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 28442624 -> 28446720 00010000:00010000:2.0:1789760798.741030:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545485ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28442624->28446719] (req 28442624->28446719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.741042:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545485ac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28442624->28446719] (req 28442624->28446719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.741051:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545485ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28442624->28446719] (req 28442624->28446719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.741057:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9290062ff600 already unused, calling callback (ffffffffc1598350) 00010000:00010000:2.0:1789760798.741062:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545485ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28442624->28446719] (req 28442624->28446719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.741070:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545485ac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28442624->28446719] (req 28442624->28446719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.741089:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545485ac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28442624->28446719] (req 28442624->28446719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.741098:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545485ac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28442624->28446719] (req 28442624->28446719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.741200:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff929018a80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:17022965-1e4f-46fb-915c-61730096b8e2+7:2357:x1876699495973376:12345-192.168.203.9@tcp:4:dd.0 Request processed in 34700us (35108us total) trans 4294974261 rc 0/0 00000100:00100000:2.0:1789760798.741215:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.9@tcp, seq: 6973 00000100:00100000:2.0:1789760798.741225:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782971611863 time 783027201676 state 0x2 ref 0 00000001:00080000:0.0:1789760798.741899:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974261 is committed 00000100:00100000:1.0:1789760798.752056:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.9@tcp (source: 12345-192.168.203.9@tcp) 00000100:00100000:3.0:1789760798.752541:0:6714:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699495973504 00000100:00100000:3.0:1789760798.752553:0:6714:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699495973504 00000100:00100000:2.0:1789760798.752756:0:6714:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 782971611863 time 783027201676 state 0x0 ref 1 00000100:00100000:2.0:1789760798.752773:0:6714:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 783027462092 time 783027201676 state 0x1 ref 1 00000100:00100000:2.0:1789760798.752805:0:6714:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff92900fde7000 key nid={192.168.203.9@tcp}&opcode={ost_write} rule@0xffff929005e88200 gen 0 rate 15 depth 3 token 3 deadline 783027462092 time 783073521567 state 0x0 ref 1 00000100:00100000:2.0:1789760798.752817:0:6714:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff928f033b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:17022965-1e4f-46fb-915c-61730096b8e2+5:2358:x1876699495973504:12345-192.168.203.9@tcp:4:dd.0 00002000:00010000:2.0:1789760798.752869:0:6714:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 28446720, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xe009 00010000:00010000:2.0:1789760798.752878:0:6714:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545485b3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.752895:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=783s v=(0 0) 00010000:00010000:2.0:1789760798.752900:0:6714:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.752906:0:6714:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545485b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28446720->28450815] (req 28446720->28450815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.752918:0:6714:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000786d1ccc/0x49367483545485b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [28446720->28450815] (req 28446720->28450815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6714 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 Debug log: 56484 lines, 56484 kept, 0 dropped, 0 bad.